Hello Guest, Welcome to Apnea Board !
As a guest, you are limited to certain areas of the board and there are some features you can't use.
To post a message, you must create a free account using a valid email address.

or Create an Account


New Posts   Today's Posts

OSCAR import of Dreem 2 CSV data
#31
RE: OSCAR import of Dreem 2 CSV data
As I've been looking at this some more, updated observation, which is probably a general issue when integrating tz-aware data with tz-naive xPAP data in OSCAR:

Quote:If one moves across timezones with one's xPAP machine, the times gets out of sync.

Dreem CSV includes the timezone info in its CSV, but the output format of this script (and I assume the Zeo format, and also therefore the OSCAR import code) is not timezone-aware.

I guess the best way to do that is to convert all datetimes to UTC, and then convert to localized default timezone at the end?

I can't yet figure out how best to do that though.
Caveats: I'm just a patient, with no medical training.
Post Reply Post Reply
#32
RE: Zeo CSV format, hypnogram import
(10-10-2019, 08:26 AM)slowriter Wrote: They say yes, it will be stable now, and that improving data export is a big focus for them.

Who were you working with? When I've asked them about data export, they've been pretty disinterested.

I'm working on a native loader for their CSV files and I'm running into a very weird situation: there are too many hypnogram samples!

For example, I have a night of 504m34s according to the start time and stop time. That *should* be 1009 or maybe 1010 samples. But the hypnogram has 1011.

I thought maybe they were using the wall clock and emitting samples at :00 and :30, as well as the end time, but that doesn't seem to match my data either.
Huh
Post Reply Post Reply
#33
RE: Zeo CSV format, hypnogram import
(01-31-2020, 12:25 PM)sawinglogz Wrote: Who were you working with? When I've asked them about data export, they've been pretty disinterested.

I'm working on a native loader for their CSV files and I'm running into a very weird situation: there are too many hypnogram samples!

For example, I have a night of 504m34s according to the start time and stop time. That *should* be 1009 or maybe 1010 samples. But the hypnogram has 1011.

I thought maybe they were using the wall clock and emitting samples at :00 and :30, as well as the end time, but that doesn't seem to match my data either.
Huh

Have you looked at the python script on github I posted earlier? It's author has some comments there that may help?

I've not had any problems with the resulting data.

I'll PM you on your first question.
Caveats: I'm just a patient, with no medical training.
Post Reply Post Reply
#34
RE: OSCAR import of Dreem 2 CSV data
Script:

https://gist.github.com/jeremyblow/542df...1284f9aec4

I forked it to add timezone support (though I don't know how well).
Caveats: I'm just a patient, with no medical training.
Post Reply Post Reply
#35
RE: Zeo CSV format, hypnogram import
(11-21-2019, 01:23 PM)sawinglogz Wrote: Could you PM me a copy of the CSV that makes OSCAR crash? We don't like crash bugs.

It turned out any CSV without all of the expected headers was causing an assertion failure.

I just merged in a new native Dreem loader, and fixed the Zeo loader in the process. The Zeo loader should now no longer crash on arbitrary CSV files, and both Zeo and Dreem hypnograms are now drawn the right way up.

They also both account for discontinuities in the data, and Zeo loader now handles sleep stage 6 ("Deep (2)" according to Zeo Viewer), which was a surprise.

And, because OSCAR can now parse the Dreem files directly, it gets the time zone information right.

...now if only I could figure out why there are too many hyponogram samples in Dreem files...
Post Reply Post Reply
#36
?
Got it built, and imported a few months of data.

First, thanks much for doing this; appreciated!

Now, a couple/few details.

Good: imports super fast, and appears to be correct. Also, thanks for fixing the display (now right-sided up).

Minor issue: the numbers on the sidebar are now -1, -2, etc. Is that necessary to get it display correctly? Because it is confusing, and conflicts with the tool tip legend.

[moving separate bug report to another thread]
Caveats: I'm just a patient, with no medical training.
Post Reply Post Reply
#37
RE: ?
(01-31-2020, 05:48 PM)slowriter Wrote: If I have focus on the OSCAR window, I can't use my Gnome keyboard shortcuts to cycle through applications.

The only way I can resolve this is if I click on the top window bar.
Caveats: I'm just a patient, with no medical training.
Post Reply Post Reply
#38
RE: ?
(01-31-2020, 05:48 PM)slowriter Wrote: Good: imports super fast, and appears to be correct. Also, thanks for fixing the display (now right-sided up).

Minor issue: the numbers on the sidebar are now -1, -2, etc. Is that necessary to get it display correctly? Because it is confusing, and conflicts with the tool tip legend.

You're exactly right. That's how I got the display to be right side up. (Upside down was driving me nuts!)


I suppose we could update the tool tip legend. I figured people would overlook the difference between the tooltip and the scale, but evidently not! Smile

...of course, if you had Zeo data, you'd also notice that there's sometimes a -3.75 for the Deep-2 stage...

In a future version we can refine some of these details, such as inverting the chart without inverting the values, and putting text labels beside it instead of numeric...

Can you start a new thread for the non-loader issues? Someone else should take a look at that.
Post Reply Post Reply
#39
RE: ?
(01-31-2020, 06:06 PM)sawinglogz Wrote:
(01-31-2020, 05:48 PM)slowriter Wrote: Good: imports super fast, and appears to be correct. Also, thanks for fixing the display (now right-sided up).

Minor issue: the numbers on the sidebar are now -1, -2, etc. Is that necessary to get it display correctly? Because it is confusing, and conflicts with the tool tip legend.

You're exactly right. That's how I got the display to be right side up. (Upside down was driving me nuts!)

I suppose we could update the tool tip legend. I figured people would overlook the difference between the tooltip and the scale, but evidently not! Smile

...of course, if you had Zeo data, you'd also notice that there's sometimes a -3.75 for the Deep-2 stage...

In a future version we can refine some of these details, such as inverting the chart without inverting the values, and putting text labels beside it instead of numeric...

Can you start a new thread for the non-loader issues? Someone else should take a look at that.

New thread started.

The "minor issue" I noted is definitely not a huge deal. I just can imagine it would confuse some people.
Caveats: I'm just a patient, with no medical training.
Post Reply Post Reply
#40
RE: OSCAR import of Dreem 2 CSV data
Regarding DREEM sleep position data, they don't seem open to putting it in their downloadable files.  I really wanted to get it though so I signed up for their research program and bought the $600 special version of their headband that permits downloading ALL the raw data. Unfortunately I didn't read all the fine print where it says that results they derive from the raw data, such as sleep position, are not available.  Since the raw data is from a 3 axis accelerometer, you have to write your own program produce sleep positions.  Not impossible of course but that means you have to also handle noise problems, ambiguities, etc and deal with the EDF format. I've put it on the back burner for now as there are more interesting things to do.
Post Reply Post Reply


Possibly Related Threads...
Thread Author Replies Views Last Post
Question OK TO IMPORT SLEEP DATA FROM OLD SD CARD INTO OSCAR W/O MESSING UP CURRENT DATA? Plmnb 1 71 6 hours ago
Last Post: Ibrumley
  Rescan app [doesn't see data import] kerm007 3 138 03-24-2024, 10:59 PM
Last Post: srlevine1
  Trouble importing data to Oscar bubbafoxus 2 86 03-22-2024, 05:40 PM
Last Post: bubbafoxus
  OSCAR Profiles and Combining Different Brand CPAP Data StratCat48 2 48 03-22-2024, 08:00 AM
Last Post: StratCat48
  Where Oscar put the data file? Rosacer24 0 58 03-22-2024, 12:16 AM
Last Post: Rosacer24
  There's 1.5 years of data in my OSCAR, but now there's a problem Snackers 8 217 03-19-2024, 03:36 AM
Last Post: Snackers
  OSCAR v1.5.2- error code 5, unable to write to data directory sleepynotion 2 106 03-18-2024, 12:09 PM
Last Post: sleepynotion


New Posts   Today's Posts


About Apnea Board

Apnea Board is an educational web site designed to empower Sleep Apnea patients.