Showing posts with label csv. Show all posts
Showing posts with label csv. Show all posts

Friday, February 22, 2013

Extracting one value from logged ascii packet data

Here is my personal trending tool for work, producing three columns space delimited file that can be imported into Excel

find . -name "*.log" -exec grep Keyword -H {} \; | sed -e 's/: 10/:10/' | awk '{print $1 " " $2 " " $68}' | sed -e 's/.l
og:2/.log: 2/' > ~/output.txt

Wednesday, October 3, 2012

IDL read csv file

My first search for this turned up more information about the function that is being used, read_csv.pro. However, all links are essentially printouts of the code, and hint that the first row must be a header row. I'm not sure if the version of this function that we have is exactly the same.

http://home.strw.leidenuniv.nl/~hoekstra/downloads/read_csv.pro

My problem is that my csv file has hexadecimal numbers, and read_csv just happens to be interpreting values with an E in the 2nd from the last digit incorrectly. I'd like it to just read in the entire file as strings and let me do the string to number conversion. There's a "STRINGS" keyword in the listing, but it's not clear how to use it. It looks like you just have to say strings=something, some examples seem to show strings=strings (it doesn't look like the /STRINGS nomenclature that goes with IDL operations).

Then, this forum thread says use readcol if you want to do everything by hand, or just use Readf or read_data_file which I guess will treat the csv file like an array:

http://www.rhinocerus.net/forum/lang-idl-pvwave/118138-reading-csv-file.html