DataSource Class Reference

DataSource is the basic object to handle input data of various sources. More...

List of all members.

Public Member Functions
bool	next ()
bool	previous ()
void	close ()
int	at ()
bool	first ()
object	value (string column_name)
bool	last ()
void	isOpen ()
string	columnName (int index)
void	open (string type, string name)
string	columnType (int index)
bool	eof ()
bool	seek (int index, bool relative)
Static Public Attributes
static int	colCount
static boolean	fixedWidth
static string	connection
static string	praefix
static bool	strictNames

---

Detailed Description

DataSource is the basic object to handle input data of various sources.

Supported formats

A DataSource is an JavaScript object that encapsulates tabular input data. Data can be loaded from various sources (e.g. text files or database columns). Supported data sources / formats are:

Text files

The DataSource is able to digest a variety of text file formats. However, the standard case is a file with columns separated by a special character and the column names in the first row. DataSource will guess the delimiter by scanning the first line. Valid delimiters are: ',', ';', "tab" (0x09) and " " (space character). Both '\n' (Unix) and '\r\n' (Windows) are valid row delimiters. Column names as well as data values may optionally be enclosed by double quotation marks ("). Additionally, lines starting with '#' are considered as comment lines and ignored.

By setting the property fixedWidth is set to true (prior the call to open()), input values are read from fixed column positions.

The presumed data types of text files are determined by scanning the first ten lines of the data.

The contents of the input file will be completely loaded into memory. For very large input files, use the format_stream option instead.

Streaming text files

This variant of Datasource is designed for streaming of (large) text files. The navigation options are limited (i.e. no seek() or previous() functions), but they can be used for processings MotiveInput.

DBF database files

The DBF support is based on a fast native C++ DBF reader that is able to read DBF files from Microsoft Excel or ESRI ArcMap.

SQL database queries

Prior to executing SQL queries, a database connections must be set up using the global function database(). The database() function returns a connection name (as a string) and database type and additional data base specific parameters. Currently, the following database formats are supported:

• SQLITE: provide the path to the sqlite database (e.g. "c:/test/mydb.sqlite")
• ODBC: provide the connection string (e.g. "Driver={Microsoft Excel Driver (*.xls)};DriverId=790;Dbq=c:\test\excel.xls;")
• MYSQL (?): not implemented (parameters: host, username, password)

The following example demonstrates how to use database connections:

var sqlite_db = database('SQLITE', 'c:/temp/mydb.sqlite'); // create a database connection (global variable)
...
function test() {
    var ds = new DataSource();
    ds.connection = sqlite_db; // set connection
    ds.open('sql', 'select * from data_table where id=2'); // complex query
    ds.open('sql', 'data_table'); // simple query; all rows from the table
    ds.showInfo(); // displays debug info about columns, ...
}

The open() method of the DataSource expects either a table name or a SQL query (see the example above). The SQL syntax rules are enforced by the underlying database engine; i.e. the syntax differs between, e.g., ODBC database and MYSQL databases.

Motive Tables

Using DataSource

Typically, a DataSource is used either for a Binding, or its data is accessed programmatically. When used in conjunction with Bindings declaratively, no explicit looping or access of individual rows is necessary. However, manual navigation is also possible:

The open() function loads the data. The parameter type indicates the source type (see Supported formats for available types). You can use the isOpen() function to check if the datasource was openend successfully. Note that there is no valid row loaded immediately after open(). Use first(), last() as well as next(), previous() to loop over the rows. Use at() to read the current row number or seek() to random-access a specific row. eof() returns false if the end of the dataset is reached. The next() function returns true as long as there are rows behind the current row; this allows a efficient loop statement as follows:

    var ds = new DataSource();
    ...
    ds.first(); // not necessary after opening
    while (ds.next()) {
       // loop over *all* records of the data set
    }

Accessing the data values of indivdiual columns by calling the value() function. In addition, also the columns of the data input itself are accessible as first-class properties. The following example illustrates this:

    // Datafile contains 2 columns: 'id' and 'value'
    var ds = new DataSource();
    ds.open();
    while (ds.next()) {
       // do something...
       print( ds.id, ds.value ); // access columns directly
       print( ds.value(0) ); // access the first column
       print( ds.value('id')); // access also the first column
    }

Handling of data types

How various data types are handled...

---

Member Function Documentation

int DataSource::at

(

)

[inline]

retrieve the index of the current record.

Returns:

The index of the current record.

void DataSource::close

(

)

[inline]

explicitely closes the data source. After calling close(), the data source must be reopenend by a call to open() before usage.

string DataSource::columnName

(

int

index

)

[inline]

retrieve the column name of column index. Column indices are 0-based. Returns an empty string if out of range.

Parameters:

index

Returns:

name of the column at index

string DataSource::columnType

(

int

index

)

[inline]

retrieve the data type of column index. Column indices are 0-based.

Parameters:

index

Returns:

Returns an empty string if out of range. See Handling of data types for details.

bool DataSource::eof

(

)

[inline]

checks, whether the end of the record set is reached.

Returns:

returns true, if index is behind the end of the valid rows.

bool DataSource::first

(

)

[inline]

loads the first record (if available) and position internal pointer to this position.

Returns:

true if operation was successful.

See also:

first(), next(), previous(), last()

void DataSource::isOpen

(

)

[inline]

returns true if the database is open, i.e. if the data source was loaded successfully.

See also:

open()

bool DataSource::last

(

)

[inline]

loads the last record and position pointer to the last record.

Returns:

true if operation was successful.

bool DataSource::next

(

)

[inline]

advance to next record and retrieve content. Returns false, if pointer is located behind the last record. The next() function provides a efficient looping construct:

// ds: a data source
while (ds.next()) {
   // do something
}

Returns:

Returns false, if pointer is located behind the last record.

void DataSource::open	(	string	type,
		string	name
	)			[inline]

opens a datasource with a given type from a specific source name. Opens a data source. Writes a output message if opening the data source failed. The following types are avaialable:

• 'txt', 'csv': 'stream': format_stream'sql': format_sql'dbf': format_dbf type Type of the datasource. name path to the file (for file based data sources) or the SQL query for database types. isOpen()

bool DataSource::previous

(

)

[inline]

go to the previous record.

Returns:

true if operation was successful.

bool DataSource::seek	(	int	index,
		bool	relative
	)			[inline]

go to a specific record within the dataset.

Parameters:

	index	row to go to. index is 0-based.
	relative	if relative is true, make a relative movement. relative is false by default, i.e. absolute addressing is the default behaviour.

Returns:

true if operation was successful.

object DataSource::value

(

string

column_name

)

[inline]

retrieve the data of the column with the name column_name of the current row.

Parameters:

column_name

name of column

Returns:

value of that column; returns a invalid object, if the data source is not opened or points to an invalid row.

---

Member Data Documentation

int DataSource::colCount [static]

Number of columns in the data source. Use to loop in conjunction with columnName() or value().

The property is read-only.

string DataSource::connection [static]

This property is to set or read the database connection for SQL data sources. See the example above. The default connection for a new DataSource is a temporary space within the SQLite database. For access to the Motive database, you can use the connection name "motive" or an empty connection name.

boolean DataSource::fixedWidth [static]

The fixedWidth property applies to CSV text input files. If true, the columns of the input file are expected to be separated by one or multiple spaces.

    function testFixedWidth()
    {
    var ds = new DataSource;
    ds.fixedWidth = true;
    ds.open('txt', 'E:/dev/motive/client/src/tests/data/fixed_width.txt');
    ds.showInfo();
    for (var i=0;i<10;i++) {
             ds.seek(i);
             print("row" + i + ": " + ds.value(7) + typeof(ds.value(7)));
       }
    }

The default value is: false

string DataSource::praefix [static]

this additional praefix is added to the names of members. In some cases this is helpful to avoid naming conflicts. The property must be set prior to opening the dataset. Note: the prafix is used only for the data source members, and not for the underlying dataset; e.g.:

 function testPraefix()
 {
     var ds = new DataSource;
     ds.praefix = "pre_";
     ds.open('txt', 'E:/dev/motive/client/src/tests/data/testset1.txt');
     ds.showInfo(); // columns do not have the praefix
     ds.first();
     print(ds.pre_id); // includes the praefix....
     }

bool DataSource::strictNames [static]

If strictNames is true (the default), calls to value() must use exactly the same, case sensitve column as in the data source. If false, the rules are relaxed: case sensitivity is not enforced, and names are valid even if provided captions are *longer* than in the data source. This can be useful, if working with data formats with truncated column names. Note: the property is only effective for calls to value(); property names of data source objects have still the full name:

 function testPraefix()
 {
     var ds = new DataSource;
     ds.strictNames = false;
     ds.open('dbf', 'test.dbf'); // assume: test.dbf has a column 'TREECOUN'
     ...
     print(ds.value('TreeCount')); // works
     print(ds.TreeCount); // undefined
     }

---

The documentation for this class was generated from the following file:

• E:/dev/motive/client/src/apidoc/datasourcejs.cpp