CSV data processing

I wrote a couple tools in perl, for CSV data processing:

dbicsv – can feed CSV data into or out from a database

csvproc – can process CSV data using a perl one-liner

Please let me know if you find these tools useful, to suggest features, or if you’re interested but don’t see how to use them.  Don’t let my humility deceive you; these are very useful tools!

Some examples:

export DB_DBMS=postgrsql DB_NAME=test
dbicsv 'select * from foo;' >foo.csv
dbicsv 'insert into foo values (?, ?, ?);' <foo.csv
dbicsv 'select * from sp_proc($Col1, $Col2);' <in.csv >out.csv

csvproc '$name = uc $name' <in.csv >out.csv
csvproc -out id,name,date 'fix_date($date)' <in.csv >out.csv
csvproc -filter '$name = lc $name; $salary >= 1e6;' <in.csv >out.csv

how to use dbicsv:

Usage: dbicsv [options] sql-query [param ...]

Runs an SQL query, returns CSV data.

  option   description            default

  -dbms ?  DBMS, determines DSN   $DB_DBMS || InterBase
  -dsn ?   DBI data source name   $DB_DSN || (from DBMS)
  -db ?    database name          $DB_NAME
  -user ?  database user          $DB_USER || $ISC_USER || "sysdba"
  -pass ?  database password      $DB_PASS || $ISC_PASSWORD || "sysdba"
  -host ?  database hostname      $DB_HOST
  -port ?  database port          $DB_PORT

  -in      input data from stdin  $DB_IN
  -head ?  column head = -|O|I|IO $DB_HEAD || IO

  -debug   show more info         $DB_DEBUG
  -help    show this message

how to use csvproc:

Usage: csvproc [options] mapping

Process CSV data with a small perl program.

  option   description

  -filter             filter lines, mapping returns boolean
  -out a,b            output fields a, b (can declare new fields)
  -help               show this message
  -debug              show debug info including the generated perl code

utility function examples:

  zero_pad($Var, 4);
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s