OSCAR import of Dreem 2 CSV data

[fenchurch]

It works, except they changed the format overnight. Integers are now self-explanatory text. Am asking them to confirm stability on this. Do we care whether integer or text?

Also, for the record, here's the zeo values:

Code:

0=null, 1=wake, 2=REM, 3=Light, 4=Deep

And result.

In my opinion, it wouldn't matter if they used integers or text, since the values themselves are not quantitative. I'm delighted to see them making changes - it's easy enough for us to accommodate, and certainly better than an unresponsive alternative. 

Great, glad to see it working. Thank you for sharing the screenshots, I was curious what the end result looked like. I'm not sure why my Daily/Overview/Statistics tabs in OSCAR are greyed out. 

Regarding the new values, we can do a map to perform the translation:

Code:

stage_map = {
   "wake": "1",
   "rem": "2",
   "light": "3",
   "deep": "4"
}
stages_in = filter(None, row.get("Hypnogram", "[]")[1:-1].split(','))
stages_out = [stage_map.get(stage.lower()) for stage in stages_in]

I updated the Gist to reflect the above change. I think we'll need to do a test to see the Dreem's behavior when there's no data. I've yet to see a null entry in the Hypnogram. I'm curious if it defaults to WAKE, null, last state, or some mysterious new value.

[slowriter]

I updated the Gist to reflect the above change. I think we'll need to do a test to see the Dreem's behavior when there's no data. I've yet to see a null entry in the Hypnogram. I'm curious if it defaults to WAKE, null, last state, or some mysterious new value.

My data has no empty values. Checking with the Dreem people and will update when I hear back.

[slowriter]

I updated the Gist to reflect the above change. I think we'll need to do a test to see the Dreem's behavior when there's no data. I've yet to see a null entry in the Hypnogram. I'm curious if it defaults to WAKE, null, last state, or some mysterious new value.

My data has no empty values. Checking with the Dreem people and will update when I hear back.

Answer: Hole?

by default it's not wake it's a hole if we have no data at all. 

[fenchurch]

I think it's safe to assume "hole" means "empty" in the set, e.g. "[WAKE,,,,,,Light,...]". I think we can accommodate that by adding an empty string to the map:

Code:

def hypnogram_to_stages(hypnogram):
   stage_map = {
       "": "0",
       "wake": "1",
       "rem": "2",
       "light": "3",
       "deep": "4",
   }
   return [stage_map.get(value.lower()) for value in hypnogram[1:-1].split(',')]

Gist is updated. Interestingly, the app showed that I had gaps in my heart rate set last night, but the hypnograph shows no empty values. This could be explained by some sensors being more sensitive to placement than others, but probably worth confirming.

[slowriter]

... Interestingly, the app showed that I had gaps in my heart rate set last night, but the hypnograph shows no empty values. This could be explained by some sensors being more sensitive to placement than others, but probably worth confirming.

I've seen the same thing, and assume as you do. 

Except I was assuming the pulse oximeter would be more straightforward to measure consistently, since it's just a single sensor, on the forehead (no hair to contend with, etc.)

You might check with their support; as you say, to confirm.

[slowriter]

Just FYI, I asked a mod to change the title to reflect a broad focus here, so easier for others (including coders) to find this.

[slowriter]

While they don't yet, I'm hoping Dreem in time will export the position data. 

Here's info on the somnopose format that OSCAR supports, for possible future reference (though yes, ideally, at some point OSCAR supports importing Dreem CSV directly).

http://www.proximalbox.com/somnopose/help/export.html

To quote key part about the columns:

The data columns are as follows:

1. Timestamp: Time and date of the data sample (as seconds since the first instant of 1 January 2001, GMT)
   
2. Orientation: Sleep position, represented by side-to-side angle of the device in degrees, -180 to 180. Zero for face up, positive values for facing right, negative values for facing left.
   
3. Inclination: Upright angle of the device (and user) in degrees, 0 to 180. Zero for an upright position (e.g., user standing), 90 for a horizonal position (user lying down), 180 for the unlikely upside-down situation.
   
4. Time_of_day: Time of the data sample as hours, minutes, and seconds, derived from the Timestamp value, using the device's time zone setting. It gives no additional information, but is provided in a human-readable format for convenient reference.
   
5. Date: Date of the data sample as year, month and day, derived from the Timestamp value. It gives no additional information, but is provided in a human-readable format for convenient reference. To reduce file size, this is not included in every row.
   

[Auriala]

I am new to using OSCAR.  I had a CPAP, and used SleepyHead.  Anyway, that's not why I am here. I have a Dreem2, have the original CSV data, and the script to change it to the dreem csv to oscar zeo, but how do I get the program to import that data?  I did see the "Data" at the top and tried to Import Zeo Data but it didn't do anything.  Is there something I am missing?  Do I need to import some old CPAP data first?

Using Windows 10, OSCAR  v1.0.1-r-1 (is there a newer update?)

[slowriter]

Sorry if a dumb question, but did you actually convert the csv data before importing?

[Auriala]

Np!  Yes. After learning where to put the conversion file and the csv file so it would convert. Then a second file showed up with the new file name. Even opened and they do look different.