Home      FAQ      Forum      Idea Exchange      Ask a Question      My Stuff      Help   
  
TeamDesk Backup and Restore Tools
DOWNLOAD LINKS
=========================

Backup, command-line version: tdbackup.zip
Backup, GUI version: tdbackupwin.zip
Restore, command-line version: tdrestore.zip
Restore, GUI version: tdrestorewin.zip
Source Code, .NET v4.5, C#: sources.zip

CHANGELOG
=========================

04/18/2017 - v2.4.1
* Fixed bug with table names containing question mark

01/25/2017 - v2.4.0
* New -x option to exclude table from backup/restore

10/06/2016 - v2.3.1
* Minor changes and fixes in error reporting

06/27/2016 - v2.3.0
* Backup retains last-modified-on-top order of records in CSV files. No changes in restore, simple version bump.

06/09/2016 - v2.2.5
* Ignore non-printable control characters while reading/writing CSV files

11/04/2015 - v2.2.4
* Recover from network loss wherever possible

09/02/2015 - v2.2.3
* Fixed .NET version requirement (tools claimed to run on v4.0 but in fact required v4.5)

06/25/2015 - v2.2.2
* Increased operation timeouts in all tools

06/25/2015 - v2.2.1
* Restore/Win now properly reports problems on an attempt to open .CSV file

06/11/2015 - v2.2.0
* Restore now checks whether the file already exists on a server and skips uploading file's data to improve performance in data roundtrip scenarios.

06/11/2015 - v2.1.0
* tdrestorewin now has an ability to restore from .CSV file directly without a map file.

06/09/2015 - v2.0.1
* Fixed problem backing up files with semicolon in the name
* Fixed folder pattern handling
* Added error/message colorization in console apps

04/06/2015 - v2.0.0
* Initial release

OVERVIEW
=========================

Backup and restore tools provided in two forms – command line version (tdbackup.exe, tdrestore.exe) is suitable for performing scheduled operations and runs under both .NET (v4.5+, preferred on Windows) and Mono (v4+, for other platforms). GUI versions of tools (tdbackupwin.exe, tdrestorewin.exe) collect parameters through user interface and run the same process as command line versions do – the input and output of command line and GUI version of a tool is identical. However, due to problems running GUI on Mono, GUI tools are limited to Windows platform only.

Backup tool stores the database data in comma-separated (.CSV) files, table by table. In addition it writes tables' structure collected during the backup into text-based .tdbackup file. Next time tool runs, it compares stored structure with actual structure. If structure was not changed, backup requests a list of records in a table and rewrites data file removing deleted records and copying not modified records as-is. Then it requests new and modified records from the server in batches by 500 records and adds them to the file and downloads file attachments. If table information or data file is missing or changed, all records are downloaded and CSV file is written anew. Will the process fail for some reason – due to network error for example, original files are left intact and the process can be safely repeated again later.

Restore works in opposite direction – it scans the file created by backup, establish the connection between CSV file and its columns and a database table and columns, splits file's records into a batches by 500 records or 20Mb and sends them to server. If some records cannot be inserted or updated (due to validation rules violation, for example), new file is written alongside original data file containing bad record's data and the error description as a last column. Original data file is left intact. You can then analyse error file, fix problems and repeat the process again.

Restore is not limited to operate in pair with backup – the file the structure is stored is text based and has quite simple structure.
* Semicolon at the beginning of the line denotes the comment
* Mappings between file and database are specified in blocks.
* First line denotes correspondence of CSV file to database table specified by its singular name
* Lines next to table map are maps of file's column to a database column
* Empty line denotes the end of the block
* There are two special column mappings:
_ * Star-to-star (* -> *) attempts to match all columns in a file to a database
_ * Column-to-empty (Column ->) ignores file's column (to override mapping created by *->*)

; Sample file

Invoices.csv -> Invoice
Invoice # -> Id
Address -> Address

Items.csv -> Item
Id -> Id
Invoice # -> Invoice

Note: do not modify .tdbackup files directly, rather copy them to a file .txt extension and do all the modifications there. Also, while backup writes column type after database column's name this information is not required for restoration process.

COMMAND LINE SWITCHES
=========================

Command line versions of backup and restore share the same set of switches
tdbackup URL switches
tdrestore URL switches

URL – is a REST API endpoint or any URL to your database or simply a database ID; all of forms listed below point to the same application

21995 (database id, https://www.teamdesk.net/secure/api/v2/ is assumed)
https://www.teamdesk.net/secure/api/v2/21995/ (REST API endpoint)
https://www.teamdesk.net/secure/api/21995/service.asmx (SOAP API endpoint)
https://www.teamdesk.net/secure/db/21995/overview.aspx?t=29323 (database URL)

Switches can be used in their full form or abbreviated to a first letter, except for -table switch that abbreviates to -a:

-u=user, -user=user
Required switch is either user's email or API token. Restore requires the user to have Manage Data setup privilege.

-p=password, -password=password
If the user is specified via email, specifies user's password. Do not use this switch with API token

-f=folder, -folder=folder (tdbackup only)
Specifies the folder to back up application data to. The folder should not contain backup files from another database.

For the folder name you may use {0:format} to include current date and time to the folder name. Format is the combination of format patterns listed below:

dd or d: The day of the month.
MM or M: The numeric month.
yyyy: The year in four digits.

-f=file-or-folder, -file=file-or-folder (tdrestore only)
The path to the file that defines mapping between file and its columns and the database table and its columns. If path points to the folder, tdrestore will search for <database-id>.tdbackup file in this folder.

-a=table-name, -table=table-name
The name of the table to backup or restore. You can use this switch multiple times, for example:
tdbackup ... -a=Invoice -a=Item (back up Invoice and Item tables)

-x=table-name, -exclude=table-name
The name of the table to exclude from backup or restore. You can use this switch multiple times, for example:
tdbackup ... -x=Invoice -x=Item (back up everything but Invoice and Item tables)

-v, -verbose
Display progress messages

-b, -break
Normally backup and restore tools attempt error recovery and continue operation whenever possible. This switch allows overriding this behavior and bailing out on first error encountered. It might be useful for problem identification.

CSV FILE-RELATED SWITCHES
=========================

-d=char, -delimiter=char
Character to use to separate columns, TAB is default.

-q=char, -quote=char
Character to use to quote the value containing the delimiter or line break. Double quote is the default.

-l=format, -linebreak=format
Format of the line break, is either CR or LF or CRLF. Latter is the default.

-e=encoding, -encoding=encoding
Text encoding name as described here (http://www.iana.org/assignments/character-sets/character-sets.xhtml ), UTF-8 is the default.
We urge you to delete all CSV files from backup folder prior to changing CSV-related switches.

CONFIGURATION FILES
=========================

All tools support configuration via XML file named after a tool with .config extension (that's tdbackup.exe.config or tdrestorewin.exe.config). File's <appConfig> section contains a set of key-value pairs. Key name is identical to a switch name without leading minus sign and the value is a value of the switch. You can use configuration file to provide shared settings for all operations (such as CSV format) and override via switches only those that differ – user account or folder to back up to.

DIFFERENCES FROM v1 TOOLS
=========================

* Backup no longer respects -type switch, though it is parsed. We've found out many people always performing full backup thinking it is safer. The assumption is wrong, in case of failure original data files are not modified with either method but doing full backup leads to unnecessary increase in database workload and network traffic. If you really want to backup anew, simply delete all .CSV files from the backup folder.

* Old backup was always writing trailing zeroes for numeric and midnight time for dates. New backup writes numeric with trailing zeroes and dates with time information stripped out.

* New backup is designed to work in a backward compatible mode – in of a new schema it looks for old backup's .schema.csv. After new backup process ends you can safely delete old schema files.

* Old restore's configuration file tdrestore.cfg specifies application URL, user's email and password inside of file. This information should be now supplied in separate tdrestore.exe.config file or via command line arguments and commented or deleted out of tdrestore.cfg file. Otherwise old and new file formats are identical except for in new format space in the beginning of column map is no longer required.

SOURCE CODE
=========================

Source code is available for download as an sources.zip archive.

Visual Studio 2013 with NuGet support is required in order to build executables.

The archive is split into five folders:

TeamDesk.BackupRestore\ - the library shared between tools. Contains classes to work with REST API, CSV files, command line and configuration handling utilities and implementation of backup and restore processes. The implementation is granular, you can choose from various entry points to customize backup and restore processes' logic for your own needs. Documentation comments are spread along code's public methods to simplify navigation.

tdbackup\ - command line backup application's entry point
tdbackupwin\ - GUI backup applications' entry point
tdrestore\ - command line restore application's entry point
tdrestorewin\ - GUI restore applications' entry point
ID
856
Category
Integration/API
Date Created
4/6/2015 8:02:58 AM
Date Updated
4/18/2017 5:54:25 AM
Related articles
Was this information helpful?
 
Back to Search Results