indian-ocean

Build Status NPM version NPM downloads js-standard-style

A NodeJS library for reading and writing data based on the file's extension. Also includes some handy file system utilities. Supports csv, tsv, json, dbf as well as custom formats and delimiters.

npm install --save indian-ocean

Example:

var io = require('indian-ocean')

var json_data = io.readDataSync('path/to/data.csv')

console.log(json_data)

/*
[
  {
    "name": "Gerald",
    "city": "Philadelphia"
  },
  {
    "name": "Marcy",
    "city": "New York"
  }
]
*/

io.writeDataSync('path/to/save/output.json', json_data, { indent: 2 })
    

API Documentation

# readers

# io.readData(filePath: String, parserOptions: (Function | Object)?, callback: Function)

source

Asynchronously read data given a path ending in the file format.

Supported formats / extensions:

  • .json Array of objects or object
  • .csv Comma-separated
  • .tsv Tab-separated
  • .psv Pipe-separated
  • .aml ArchieML
  • .txt Text file (a string)
  • .dbf Database format used for shapefiles
  • other All others are read as a text file
Parameters
filePath (String) Input file path
parserOptions ((Function | Object)?) Optional map function or an object specifying the optional options below.
Name Description
parserOptions.parser (String | Function | Object)? This can be a string that is the file's delimiter, a function that returns JSON, or, for convenience, can also be a dsv object such as dsv.dsv('_') or any object that has a parse method that's a function. See parsers in library source for examples.
parserOptions.map Function? Transformation function. See directReaders for format-specific function signature. In brief, tabular formats get passed a (row, i, columns) and must return the modified row. Text or AML formats are passed the full document and must return the modified document. JSON arrays are mapped like tabular documents with (row, i) and return the modified row. JSON objects are mapped with Underscore's _.mapObject with (value, key) and return the modified value.
parserOptions.reviver Function? Used for JSON files, otherwise ignored. See readJson for details.
parserOptions.filename Function? Used for JSON files, otherwise ignored. See readJson for details.
callback (Function) Has signature (err, data)
Example
io.readData('path/to/data.tsv', function (err, data) {
  console.log(data) // Json data
})

// Parser specified as a string
io.readData('path/to/data.usv', {parser: '_'}, function (err, data) {
  console.log(data) // Json data
})

// Parser specified as a function
var myParser = dsv.dsv('_').parse
// var myParser = dsv.dsv('_') // This also works
io.readData('path/to/data.usv', {parser: myParser}, function (err, data) {
  console.log(data) // Json data
})

// Parser specified as a function
var naiveJsonLines = function (dataAsString) {
  return dataAsString.split('\n').map(function (row) { return JSON.parse(row) })
}
io.readData('path/to/data.jsonlines', {parser: naiveJsonLines}, function (err, data) {
  console.log(data) // Json data
})

// Shorthand for specifying a map function
io.readData('path/to/data.csv', function (row, i, columns) {
  console.log(columns) // [ 'name', 'occupation', 'height' ]
  row.height = +row.height // Convert this value to a number
  return row
}, function (err, data) {
  console.log(data) // Json data
})

// Explicitly specify a map function and a filename for a json file. See `readJson` for more details
io.readData('path/to/data.json', {
  map: function (k, v) {
    if (typeof v === 'number') {
      return v * 2
    }
    return v
  },
  filename: 'awesome-data.json'
}, function (err, data) {
  console.log(data) // Json data with any number values multiplied by two and errors reported with `fileName`
})

# io.readDataSync(filePath: String, parserOptions: (Function | Object)?)

source

Syncronous version of readData. Read data given a path ending in the file format. This function detects the same formats as the asynchronous readData except for .dbf files, which it cannot read.

  • .json Array of objects or object
  • .csv Comma-separated
  • .tsv Tab-separated
  • .psv Pipe-separated
  • .aml ArchieML
  • .txt Text file (a string)
  • other All others are read as a text file
Returns
Object: the contents of the file as JSON
Parameters
filePath (String) Input file path
parserOptions ((Function | Object)?) Optional map function or an object specifying the optional options below.
Name Description
parserOptions.parser (String | Function | Object)? This can be a string that is the file's delimiter, a function that returns JSON, or, for convenience, can also be a dsv object such as dsv.dsv('_') or any object that has a parse method that's a function. See parsers in library source for examples.
parserOptions.map Function? Transformation function. See directReaders for format-specific function signature. In brief, tabular formats get passed a (row, i, columns) and must return the modified row. Text or AML formats are passed the full document and must return the modified document. JSON arrays are mapped like tabular documents with (row, i) and return the modified row. JSON objects are mapped with Underscore's _.mapObject with (value, key) and return the modified value.
parserOptions.reviver Function? Used for JSON files, otherwise ignored. See readJsonSync for details.
parserOptions.filename Function? Used for JSON files, otherwise ignored. See readJsonSync for details.
Example
var data = io.readDataSync('path/to/data.tsv')
console.log(data) // Json data

// Parser specified as a string
var data = io.readDataSync('path/to/data.usv', {parser: '_'})
console.log(data) // Json data

// Parser specified as a function
var myParser = dsv.dsv('_').parse
// var myParser = dsv.dsv('_') // This also works
var data = io.readDataSync('path/to/data.usv', {parser: myParser})
console.log(data) // Json data

// Parser as an object with a `parse` method
var naiveJsonLines = function(dataAsString) {
  return dataAsString.split('\n').map(function (row) { return JSON.parse(row) })
}
var data = io.readDataSync('path/to/data.jsonlines', {parser: naiveJsonLines})
console.log(data) // Json data

// Shorthand for specifying a map function
var data = io.readData('path/to/data.csv', function (row, i, columns) {
  console.log(columns) // [ 'name', 'occupation', 'height' ]
  row.height = +row.height // Convert this value to a number
  return row
})
console.log(data) // Json data

// Explicitly specify a map function and a filename for a json file. See `readJson` for more details
var data = io.readData('path/to/data.json', {
  map: function (k, v) {
    if (typeof v === 'number') {
      return v * 2
    }
    return v
  },
  filename: 'awesome-data.json'
})
console.log(data) // Json data with any number values multiplied by two and errors reported with `fileName`

# io.readdirFilter(dirPath: String, options: Object, callback: Function)

source

Asynchronously get a list of a directory's files and folders if certain critera are met.

Parameters
dirPath (String) The directory to read from
options (Object) Filter options, see below
Name Description
options.fullPath Boolean (default false) If true , return the full path of the file, otherwise just return the file name.
options.skipFiles Boolean (default false) If true , omit files from results.
options.skipDirs Boolean (default false) If true , omit directories from results.
options.skipHidden Boolean (default false) If true , omit files that start with a dot from results. Shorthand for {exclude: /^\./} .
options.include (String | RegExp | Array<(String | RegExp)>) If given a string, return files that have that string as their extension. If given a Regular Expression, return the files whose name matches the pattern. Can also take a list of either type. List matching behavior is described in includeMatchAll .
options.exclude (String | RegExp | Array<(String | RegExp)>) If given a string, return files that do not have that string as their extension. If given a Regular Expression, omit files whose name matches the pattern. Can also take a list of either type. List matching behavior is described in excludeMatchAll .
options.includeMatchAll Boolean (default false) If true, require all include conditions to be met for a file to be included.
options.excludeMatchAll Boolean (default false) If true, require all exclude conditions to be met for a file to be excluded.
callback (Function) Has signature (err, data) where files is a list of matching file names.
Example
// dir contains `data-0.tsv`, `data-0.json`, `data-0.csv`, `data-1.csv`, `.hidden-file`
io.readdirFilter('path/to/files', {include: 'csv'}, function(err, files){
  console.log(files) // ['data-0.csv', 'data-1.csv']
})

io.readdirFilter('path/to/files', {include: [/^data/], exclude: ['csv', 'json']}, , function(err, files){
  console.log(files) // ['path/to/files/data-0.csv', 'path/to/files/data-1.csv', 'path/to/files/data-0.tsv']
})

# io.readdirFilterSync(dirPath: String, options: Object)

source

Syncronous version of readdirFilter. Get a list of a directory's files and folders if certain critera are met.

Returns
Array<String>: List of matching file names
Parameters
dirPath (String) The directory to read from
options (Object) Filter options, see below
Name Description
options.fullPath Boolean (default false) If true , return the full path of the file, otherwise just return the file name.
options.skipFiles Boolean (default false) If true , omit files from results.
options.skipDirs Boolean (default false) If true , omit directories from results.
options.skipHidden Boolean (default false) If true , omit files that start with a dot from results. Shorthand for {exclude: /^\./} .
options.include (String | RegExp | Array<(String | RegExp)>) If given a string, return files that have that string as their extension. If given a Regular Expression, return the files whose name matches the pattern. Can also take a list of either type. List matching behavior is described in includeMatchAll .
options.exclude (String | RegExp | Array<(String | RegExp)>) If given a string, return files that do not have that string as their extension. If given a Regular Expression, omit files whose name matches the pattern. Can also take a list of either type. List matching behavior is described in excludeMatchAll .
options.includeMatchAll Boolean (default false) If true, require all include conditions to be met for a file to be included.
options.excludeMatchAll Boolean (default false) If true, require all exclude conditions to be met for a file to be excluded.
Example
// dir contains `data-0.tsv`, `data-0.json`, `data-0.csv`, `data-1.csv`, `.hidden-file`
var files = io.readdirFilterSync('path/to/files', {include: 'csv'})
console.log(files) // ['data-0.csv', 'data-1.csv']

var files = io.readdirFilterSync('path/to/files', {include: [/^data/], exclude: 'json', fullPath: true})
console.log(files) // ['path/to/files/data-0.csv', 'path/to/files/data-1.csv', 'path/to/files/data-0.tsv']

var files = io.readdirFilterSync('path/to/files', {include: [/^data/, 'json'], fullPath: true, includeMatchAll: true})
console.log(files) // ['path/to/files/data-0.json', 'path/to/files/data-1.json']

# writers

# io.writeData(filePath: String, data: (Array | Object | String), options: Object?, callback: Function)

source

Write the data object, inferring the file format from the file ending specified in fileName.

Supported formats:

  • .json Array of objects, also supports .geojson and .topojson
  • .csv Comma-separated
  • .tsv Tab-separated
  • .psv Pipe-separated
  • .dbf Database file, commonly used in ESRI-shapefile format.
Parameters
filePath (String) Input file path
data ((Array | Object | String)) Data to write
options (Object?) Optional options object, see below
Name Description
options.makeDirectories Boolean (default false) If true , create intermediate directories to your data file. Can also be makeDirs for short.
options.replacer (Function | Array)? Used for JSON formats. Function to filter your objects before writing or an array of whitelisted keys to keep. Examples below. See JSON.stringify docs for more info https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify
options.verbose Boolean (default true) Verbose logging output. Currently, the only logging output is a warning if you write an empty file. Set to false if don't want that.
options.indent Number? Used for JSON format. Specifies indent level. Default is 0 .
options.columns Array? Used for tabular formats. Optionally specify a list of column names to use. Otherwise they are detected from the data. See d3-dsv for more detail: https://github.com/d3/d3-dsv/blob/master/README.md#dsv_format
callback (Function) Has signature (err, dataStr) . dataStr is the data that was written out as a string
Example
io.writeData('path/to/data.json', jsonData, function (err, dataString) {
  console.log(err)
})

io.writeData('path/to/create/to/data.csv', flatJsonData, {makeDirectories: true}, function (err, dataString) {
  console.log(err)
})

io.writeData('path/to/to/data.json', jsonData, {indent: 4}, function (err, dataString) {
  console.log(err)
})

io.writeData('path/to/to/data.json', jsonData, {
  indent: 4,
  replacer: function (key, value) {
    // Filtering out string properties
    if (typeof value === "string") {
      return undefined
    }
    return value
  }
}, function (err, dataString) {
  console.log(err)
})

io.writeData('path/to/to/data.json', jsonData, {
  indent: 4,
  replacer: ['name', 'occupation'] // Only keep "name" and "occupation" values
}, function (err, dataString) {
  console.log(err)
})

# io.writeDataSync(filePath: String, data: (Array | Object | String), options: Object?)

source

Syncronous version of writers#writeData

Supports the same formats with the exception of .dbf files

Returns
String: The that was written as a string
Parameters
filePath (String) Input file path
data ((Array | Object | String)) Data to write
options (Object?) Optional options object, see below
Name Description
options.makeDirectories Boolean (default false) If true , create intermediate directories to your data file. Can also be makeDirs for short.
options.replacer (Function | Array)? Used for JSON formats. Function to filter your objects before writing or an array of whitelisted keys to keep. Examples below. See JSON.stringify docs for more info https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify
options.verbose Boolean (default true) Verbose logging output. Currently, the only logging output is a warning if you write an empty file. Set to false if don't want that.
options.indent Number? Used for JSON format. Specifies indent level. Default is 0 .
options.columns Array? Used for tabular formats. Optionally specify a list of column names to use. Otherwise they are detected from the data. See d3-dsv for more detail: https://github.com/d3/d3-dsv/blob/master/README.md#dsv_format
Example
io.writeDataSync('path/to/data.json', jsonData)

io.writeDataSync('path/to/create/to/data.csv', flatJsonData, {makeDirs: true})

io.writeDataSync('path/to/to/data.json', jsonData, {indent: 4})

io.writeDataSync('path/to/to/data.json', jsonData, {
  indent: 4,
  replacer: function (key, value) {
    // Filtering out string properties
    if (typeof value === "string") {
      return undefined
    }
    return value
  }
})

io.writeDataSync('path/to/to/data.json', jsonData, {
  indent: 4,
  replacer: ['name', 'occupation'] // Only keep "name" and "occupation" values
})

# io.appendData(filePath: String, data: (Array | Object), options: Object?, callback: Function)

source

Append to an existing data object, creating a new file if one does not exist. If appending to an object, data is extended with Object.assign. For tabular formats (csv, tsv, etc), existing data and new data must be an array of flat objects (cannot contain nested objects or arrays).

Supported formats:

  • .json Array of objects
  • .csv Comma-separated
  • .tsv Tab-separated
  • .psv Pipe-separated

Note: Does not currently support .dbf files.

Parameters
filePath (String) File to append to
data ((Array | Object)) The new data to append
options (Object?) Optional options object passed to writeData . See that function for format-specific options.
callback (Function) Has signature (err, data) . Data is the combined data that was written out
Example
io.appendData('path/to/data.json', jsonData, function (err) {
  console.log(err)
})

io.appendData('path/to/create/to/data.csv', flatJsonData, {makeDirectories: true}, function (err){
  console.log(err)
})

# io.appendDataSync(filePath: String, data: (Array | Object), options: Object?)

source

Synchronous version of writers#appendData. See that function for supported formats

Returns
Object: The combined data that was written
Parameters
filePath (String) File to append to
data ((Array | Object)) The new data to append
options (Object?) Optional options object passed to writeData . See that function for format-specific options.
Example
io.appendDataSync('path/to/data.json', jsonData)

io.appendDataSync('path/to/create/to/data.csv', flatJsonData, {makeDirectories: true})

# directReaders

# io.readAml(filePath: String, map: (Function | Object)?, callback: Function)

source

Asynchronously read an ArchieMl file. Returns an empty object if file is empty.

Parameters
filePath (String) Input file path
map ((Function | Object)?) Optional map function or an object with map key that is a function. Takes the parsed file (usually an object) and must return the modified file. See example below.
callback (Function) Has signature (err, data)
Example
io.readAml('path/to/data.aml', function (err, data) {
  console.log(data) // json data
})

// With map
io.readAml('path/to/data.aml', function (amlFile) {
  amlFile.height = amlFile.height * 2
  return amlFile
}, function (err, data) {
  console.log(data) // json data with height multiplied by 2
})

# io.readAmlSync(filePath: String, map: Function?)

source

Synchronously read an ArchieML file. Returns an empty object if file is empty.

Returns
Object: The parsed file
Parameters
filePath (String) Input file path
map (Function?) Optional map function. Takes the parsed file (usually an object) and must return the modified file. See example below.
Example
var data = io.readAmlSync('path/to/data.aml')
console.log(data) // json data

var data = io.readAmlSync('path/to/data-with-comments.aml', function (amlFile) {
  amlFile.height = amlFile.height * 2
  return amlFile
})
console.log(data) // json data with height multiplied by 2

# io.readDbf(filePath: String, map: (Function | Object)?, callback: Function)

source

Asynchronously read a dbf file. Returns an empty array if file is empty.

Parameters
filePath (String) Input file path
map ((Function | Object)?) Optional map function or an object with map key that is a function. Called once for each row with the signature (row, i) and must return the transformed row. See example below.
callback (Function) Has signature (err, data)
Example
io.readDbf('path/to/data.dbf', function (err, data) {
  console.log(data) // Json data
})

// Transform values on load
io.readDbf('path/to/data.csv', function (row, i) {
  console.log(columns) // [ 'name', 'occupation', 'height' ]
  row.height = +row.height // Convert this value to a number
  return row
}, function (err, data) {
  console.log(data) // Converted json data
})

# io.readCsv(filePath: String, map: (Function | Object)?, callback: Function)

source

Asynchronously read a comma-separated value file. Returns an empty array if file is empty.

Parameters
filePath (String) Input file path
map ((Function | Object)?) Optional map function or an object with map key that is a function. Called once for each row with the signature (row, i) and must return the transformed row. See example below or d3-dsv documentation for details.
callback (Function) Has signature (err, data)
Example
io.readCsv('path/to/data.csv', function (err, data) {
  console.log(data) // Json data
})

// Transform values on load
io.readCsv('path/to/data.csv', function (row, i, columns) {
  console.log(columns) // [ 'name', 'occupation', 'height' ]
  row.height = +row.height // Convert this value to a number
  return row
}, function (err, data) {
  console.log(data) // Converted json data
})

// Pass in an object with a `map` key
io.readCsv('path/to/data.csv', {map: function (row, i, columns) {
  console.log(columns) // [ 'name', 'occupation', 'height' ]
  row.height = +row.height // Convert this value to a number
  return row
}}, function (err, data) {
  console.log(data) // Converted json data
})

# io.readCsvSync(filePath: String, map: (Function | Object)?)

source

Synchronously read a comma-separated value file. Returns an empty array if file is empty.

Returns
Array: the contents of the file as JSON
Parameters
filePath (String) Input file path
map ((Function | Object)?) Optional map function or an object with map key that is a function. Called once for each row with the signature (row, i) and must return the transformed row. See example below or d3-dsv documentation for details.
Example
var data = io.readCsvSync('path/to/data.csv')
console.log(data) // Json data

// Transform values on load
var data = io.readCsvSync('path/to/data.csv', function (row, i, columns) {
  console.log(columns) // [ 'name', 'occupation', 'height' ]
  row.height = +row.height // Convert this value to a number
  return row
})
console.log(data) // Json data with casted values

// Pass in an object with a `map` key
var data = io.readCsvSync('path/to/data.csv', {map: function (row, i, columns) {
  console.log(columns) // [ 'name', 'occupation', 'height' ]
  row.height = +row.height // Convert this value to a number
  return row
}})
console.log(data) // Json data with casted values

# io.readJson(filePath: String, parserOptions: (Function | Object)?, callback: Function)

source

Asynchronously read a JSON file. Returns an empty array if file is empty.

Parameters
filePath (String) Input file path
parserOptions ((Function | Object)?) Optional map function or an object specifying the optional options below.
Name Description
parserOptions.map Function? Map function. Called once for each row if your file is an array (it tests if the first non-whitespace character is a [ ) with a callback signature (row, i) and delegates to _.map . Otherwise it's considered an object and the callback the signature is (value, key) and delegates to _.mapObject . See example below.
parserOptions.filename String? File name displayed in the error message.
parserOptions.reviver Function? A function that prescribes how the value originally produced by parsing is mapped before being returned. See JSON.parse docs for more: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse#Using_the_reviver_parameter
callback (Function) Has signature (err, data)
Example
io.readJson('path/to/data.json', function (err, data) {
  console.log(data) // Json data
})

// Specify a map
io.readJson('path/to/data.json', function (row, i) {
  row.height = row.height * 2
  return row
}, function (err, data) {
  console.log(data) // Json data with height multiplied by two
})

// Specify a filename
io.readJson('path/to/data.json', 'awesome-data.json', function (err, data) {
  console.log(data) // Json data, any errors are reported with `fileName`.
})

// Specify a map and a filename
io.readJson('path/to/data.json', {
  map: function (row, i) {
    row.height = row.height * 2
    return row
  },
  filename: 'awesome-data.json'
}, function (err, data) {
  console.log(data) // Json data with any number values multiplied by two and errors reported with `fileName`
})

// Specify a map and a filename on json object
io.readJson('path/to/json-object.json', {
  map: function (value, key) {
    if (typeof value === 'number') {
      return value * 2
    }
    return value
  },
  filename: 'awesome-data.json'
}, function (err, data) {
  console.log(data) // Json data with any number values multiplied by two and errors reported with `fileName`
})

// Specify a reviver function and a filename
io.readJson('path/to/data.json', {
  reviver: function (k, v) {
    if (typeof v === 'number') {
      return v * 2
    }
    return v
  },
  filename: 'awesome-data.json'
}, function (err, data) {
  console.log(data) // Json data with any number values multiplied by two and errors reported with `fileName`
})

# io.readJsonSync(filePath: String, parserOptions: (Function | Object)?)

source

Synchronously read a JSON file. Returns an empty array if file is empty.

Returns
(Array | Object): The contents of the file as JSON
Parameters
filePath (String) Input file path
parserOptions ((Function | Object)?) Optional map function or an object specifying the optional options below.
Name Description
parserOptions.map Function? Map function. Called once for each row if your file is an array (it tests if the first non-whitespace character is a [ ) with a callback signature (row, i) and delegates to _.map . Otherwise it's considered an object and the callback the signature is (value, key) and delegates to _.mapObject . See example below.
parserOptions.filename String? File name displayed in the error message.
parserOptions.reviver Function? A function that prescribes how the value originally produced by parsing is mapped before being returned. See JSON.parse docs for more: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse#Using_the_reviver_parameter
Example
var data = io.readJsonSync('path/to/data.json')
console.log(data) // Json data

// Specify a map
var data = io.readJson('path/to/data.json', function (k, v) {
  if (typeof v === 'number') {
    return v * 2
  }
  return v
})
console.log(data) // Json data with any number values multiplied by two

// Specify a filename
var data = io.readJson('path/to/data.json', 'awesome-data.json')
console.log(data) // Json data, any errors are reported with `fileName`.

// Specify a map and a filename
var data = io.readJsonSync('path/to/data.json', {
  map: function (k, v) {
    if (typeof v === 'number') {
      return v * 2
    }
    return v
  },
  filename: 'awesome-data.json'
})

console.log(data) // Json data with any number values multiplied by two and errors reported with `fileName`

# io.readPsv(filePath: String, map: (Function | Object)?, callback: Function)

source

Asynchronously read a pipe-separated value file. Returns an empty array if file is empty.

Parameters
filePath (String) Input file path
map ((Function | Object)?) Optional map function or an object with map key that is a function. Called once for each row with the signature (row, i) and must return the transformed row. See example below or d3-dsv documentation for details.
callback (Function) Has signature (err, data)
Example
io.readPsv('path/to/data.psv', function (err, data) {
  console.log(data) // Json data
})

// Transform values on load
io.readPsv('path/to/data.psv', function (row, i, columns) {
    console.log(columns) // [ 'name', 'occupation', 'height' ]
    row.height = +row.height // Convert this value to a number
    return row
}, function (err, data) {
  console.log(data) // Json data with casted values
})

# io.readPsvSync(filePath: String, map: (Function | Object)?)

source

Synchronously read a pipe-separated value file. Returns an empty array if file is empty.

Returns
Array: The contents of the file as JSON
Parameters
filePath (String) Input file path
map ((Function | Object)?) Optional map function or an object with map key that is a function. Called once for each row with the signature (row, i) and must return the transformed row. See example below or d3-dsv documentation for details.
Example
var data = io.readPsvSync('path/to/data.psv')
console.log(data) // Json data

var data = io.readPsvSync('path/to/data.psv', function (row, i, columns) {
  console.log(columns) // [ 'name', 'occupation', 'height' ]
  row.height = +row.height // Convert this value to a number
  return row
})
console.log(data) // Json data with casted values

# io.readTsv(filePath: String, map: (Function | Object)?, callback: Function)

source

Asynchronously read a tab-separated value file. Returns an empty array if file is empty.

Parameters
filePath (String) Input file path
map ((Function | Object)?) Optional map function or an object with map key that is a function. Called once for each row with the signature (row, i) and must return the transformed row. See example below or d3-dsv documentation for details.
callback (Function) Has signature (err, data)
Example
io.readTsv('path/to/data.tsv', function (err, data) {
  console.log(data) // Json data
})

// Transform values on load
io.readTsv('path/to/data.tsv', function (row, i, columns) {
    console.log(columns) // [ 'name', 'occupation', 'height' ]
    row.height = +row.height // Convert this value to a number
    return row
}, function (err, data) {
  console.log(data) // Json data with casted values
})

# io.readTsvSync(filePath: String, map: Function?)

source

Synchronously read a tab-separated value file. Returns an empty array if file is empty.

Returns
Array: the contents of the file as JSON
Parameters
filePath (String) Input file path
map (Function?) Optional map function, called once for each row (header row skipped). Has signature (row, i, columns) and must return the transformed row. See example below or d3-dsv documentation for details.
Example
var data = io.readTsvSync('path/to/data.tsv')
console.log(data) // Json data

// Transform values on load
var data = io.readTsvSync('path/to/data.tsv', function (row, i, columns) {
  console.log(columns) // [ 'name', 'occupation', 'height' ]
  row.height = +row.height // Convert this value to a number
  return row
})
console.log(data) // Json data with casted values

# io.readTxt(filePath: String, map: (Function | Object)?, callback: Function)

source

Asynchronously read a text file. Returns an empty string if file is empty.

Parameters
filePath (String) Input file path
map ((Function | Object)?) Optional map function or an object with map key that is a function. Takes the file read in as text and return the modified file. See example below.
callback (Function) Has signature (err, data)
Example
io.readTxt('path/to/data.txt', function (err, data) {
  console.log(data) // string data
})

io.readTxt('path/to/data.txt', function (str) {
  return str.replace(/hello/g, 'goodbye') // Replace all instances of `"hello"` with `"goodbye"`
}, function (err, data) {
  console.log(data) // string data with values replaced
})

# io.readTxtSync(filePath: String, map: (Function | Object)?)

source

Synchronously read a text file. Returns an empty string if file is empty.

Returns
String: the contents of the file as a string
Parameters
filePath (String) Input file path
map ((Function | Object)?) Optional map function or an object with map key that is a function. Takes the file read in as text and must return the modified file. See example below.
Example
var data = io.readTxtSync('path/to/data.txt')
console.log(data) // string data

var data = io.readTxtSync('path/to/data.txt', function (str) {
  return str.replace(/hello/g, 'goodbye') // Replace all instances of `"hello"` with `"goodbye"`
})
console.log(data) // string data with values replaced

# converters

# io.convertData(inFilePath: String, outFilePath: String, options: Object?, callback: Function)

source

Reads in data given a path ending in the file format with readData and writes to file using writeData. A convenience function for converting files to more other formats. All formats can convert to all others except as long as they are lists. For example, you can't convert an object-based aml file to a list format.

Parameters
inFilePath (String) Input file path
outFilePath (String) Output file path
options (Object?) Optional config object that's passed to writeData . See that documentation for full options, which vary depending on the output format you choose.
callback (Function) Has signature (err, dataStr) . dataStr is the data that was written out as a string
Example
io.convertData('path/to/data.dbf', 'path/to/data.csv', function (err, dataStr) {
  console.log(err)
})

io.convertData('path/to/data.tsv', 'path/to/create/to/data.dbf', {makeDirectories: true}, function (err, dataStr) {
  console.log(err)
})

# io.convertDbfToData(inFilePath: String, outFilePath: String, options: Object?, callback: Function)

source

Reads in a dbf file with readData and write to file using writeData. A convenience function for converting DBFs to more useable formats. Formerly known as writeDbfToData and is aliased for legacy support.

Parameters
inFilePath (String) Input file path
outFilePath (String) Output file path
options (Object?) Optional config object that's passed to writeData . See that documentation for full options, which vary depending on the output format you choose.
callback (Function) Has signature (err, dataStr) . dataStr is the data that was written out as a string
Example
io.convertDbfToData('path/to/data.dbf', 'path/to/data.csv', function (err, dataStr) {
  console.log(err)
})

io.convertDbfToData('path/to/data.dbf', 'path/to/create/to/data.csv', {makeDirectories: true}, function (err, dataStr) {
  console.log(err)
})

# helpers

# io.discernFileFormatter(filePath: String)

source

Returns a formatter that will format json data to file type specified by the extension in filePath. Used internally by writeData and writeDataSync.

Returns
Function: A formatter function that will write the extension format
Parameters
filePath (String) Input file path
Example
var formatter = io.discernFileFormatter('path/to/data.tsv')
var csv = formatter(json)

# io.discernFormat(filePath: String)

source

Given a filePath return the file's extension. Used internally by discernParser and discernFileFormatter. Returns false for files without an extension, including dotfiles

Returns
String: The file's extension
Parameters
filePath (String) Input file path
Example
var format = io.discernFormat('path/to/data.csv')
console.log(format) // 'csv'
var format = io.discernFormat('path/to/.dotfile')
console.log(format) // false

# io.discernParser(filePath: String?, options: Object?)

source

Given a filePath return a parser that can read that file as json. Parses as text if format not supported by a built-in parser. If given a delimiter string as the second argument, return a parser for that delimiter regardless of filePath. Used internally by readData and readDataSync.

Returns
Function: A parser that can parse a file string into json
Parameters
filePath (String?) Input file path
options (Object?) Optional options object, see below
Name Description
options.delimiter Object? If {delimiter: true} , it will treat the string given as filePath as a delimiter and delegate to dsv.dsvFormat .
Example
var parser = io.discernParser('path/to/data.csv')
var json = parser('name,price\nApple,120\nPear,300')

var parser = io.discernParser('_', {delimiter: true})
var json = parser('name_price\nApple_120\nPear_300')

# io.exists(filePath: String, callback: Function)

source

Asynchronously test whether a file exists or not by using fs.access modified from https://github.com/nodejs/io.js/issues/1592#issuecomment-98392899.

Parameters
filePath (String) Input file path
callback (Function) Has signature (err, exists)
Example
var exists = io.exists('path/to/data.tsv', function (err, exists) {
  console.log(exists) // `true` if the file exists, `false` if not.
})

# io.existsSync(filePath: String)

source

Syncronous version of exists. Delegates to fs.existsSync if that function is available.

Returns
Boolean: Whether the file exists or not
Parameters
filePath (String) Input file path
Example
var exists = io.existsSync('path/to/data.tsv')
console.log(exists) // `true` if file exists, `false` if not.

# io.extMatchesStr(filePath: String, extension: String)

source

Test whether a file name has the given extension

Returns
Boolean: Whether it matched or not.
Parameters
filePath (String) Input file path
extension (String) The extension to test. An empty string will match a file with no extension.
Example
var matches = io.extMatchesStr('path/to/data.tsv', 'tsv')
console.log(matches) // `true`

# io.makeDirectories(outPath: String, callback: Function)

source

Asynchronously create directories along a given file path. Delegates to mkdirp module. If the last element in your file path is also a folder, it must end in / or else it will be interpreted as a file and not created.

Parameters
outPath (String) The path to a file
callback (Function) The function to do once this is done. Has signature of (err)
Example
io.makeDirectories('path/to/create/to/data.tsv', function (err) {
  console.log(err) // null
})

// Must end in `/` for the last item to be interpreted as a folder as well.
io.makeDirectories('path/to/create/to/another-folder/', function (err) {
  console.log(err) // null
})

# io.makeDirectoriesSync(outPath: String)

source

Synchronous version of makeDirectories. Delegates to mkdirp module.

Parameters
outPath (String) The path to a file
Example
io.makeDirectoriesSync('path/to/create/to/data.tsv')
// Must end in `/` for the last item to be interpreted as a folder as well.
io.makeDirectoriesSync('path/to/create/to/another-folder/')

# io.matches(filePath: String)

source

Test whether a file name or path matches a given matcher. Delegates to extMatchesStr if matcher is a string` and tests only against the file name extension. Delegates to matchesRegExp if matcher is a Regular Expression and tests against entire string, which is usefulf or testing the full file path.

Returns
(String | RegExp): matcher The string or Regular Expression to match against.
Parameters
filePath (String) Input file path or path to the file.
Example
var matches = io.matches('path/to/data.tsv', 'tsv')
console.log(matches) // `true`

var matches = io.matches('.gitignore', /\.gitignore/)
console.log(matches) // `true`

var matches = io.matches('file/with/no-extention', '') // Nb. Dot files are treated as files with no extention
console.log(matches) // `true`

# io.matchesRegExp(filePath: String, RegExp: RegExp)

source

Test whether a string matches a given Regular Expression.

Returns
Boolean: Whether they match.
Parameters
filePath (String) Input file path or file path.
RegExp (RegExp) The Regular Expression to match against.
Example
var matches = io.matchesRegExp('.gitignore', /\.gitignore/)
console.log(matches) // `true`

var matches = io.matchesRegExp('data/final-data/basic.csv', /\/final-data\//)
console.log(matches) // `true`