Re: Loading Files with mixed text and numbers

[Przemek Klosowski]

On 07/09/2017 06:53 PM, siko1056 wrote:

Fritz Sonnichsen wrote

I did this all the time with Matlab--loaded files with the very common 
form of records like this:
2017/07/07,   13:59:59,   022.2,   12.69

Personally, I found that it's a little awkward to process dates in Octave, so I like to keep my time series data in sqlite because it has a builtin CSV import, and does dates very well. The purist may object to such mixing, but the practical person would point out that the Domain Specific Languages (DSL) concept is a cool idea.

For Octave use, it's convenient to convert dates to numbers, e.g. Julian day numbers used by astronomers.  I'd do something like

sqlite3 mydata.db -csv '.import mydata.csv m' # import the CSV file to table 'm'

sqlite3 mydata.db 'select julianday(date),col3,col4 from m'   # spit out the data

or actually use the sqlite extension in Octave to read the database directly instead of using []=system()

Caveats:

- it's easier if the first line of the CSV file has column headers---they become the SQL column names

- actually, since your dates use / instead of - for separator, and separates date and time, and doesn't specify timezone, it'd be

sqlite3 mydata.db 'select julianday(replace(date,"/","-")||" "||time,"localtime"),col3,col4 from m'