Changelog for Release 0.7.0-alpha (07/14/2015)

• Provides a new command-line interface deepdive with a new standard DeepDive application layout.

  • No more installation/configuration complication: Users run everything through the only deepdive command, and everything just works in any environment. The only possible failure mode is not being able to run deepdive command, e.g., by not setting up the PATH environment correctly.
  • No more pathname/environment clutter in apps: repeated settings for DEEPDIVE_HOME, APP_HOME, PYTHONPATH, LD_LIBRARY_PATH, PGHOST, PGPORT, ... in run.sh or env.sh or env_local.sh or env_db.sh or etc. are gone. Path names (e.g., extractor udf) in application.conf are all relative to the application root, and brittle relative paths are no longer used in any of the examples.
  • Clear separation of app code from infrastructure code, as well as source code from object code: No more confusing of deepdive source tree with binary/executable/shared-library distribution or temporary/log/output directories.
  • Binary releases can be built with make package.

• Here are a summary of changes visible to users:

  • Application settings is now kept in deepdive.conf file instead of application.conf.
  • Database settings is now done by putting everything (host, port, user, password, database name) into a single URL in file db.url.
  • Path names (e.g., extractor udf) in deepdive.conf are all relative to the application root unless they are absolute paths.
  • SQL queries against the database can be run easily with deepdive sql command when run under an application.
  • Database schema is now put in file schema.sql and optional initial data loading can be done by a script input/init.sh. Input data is recommended to be kept under input/.
  • By passing the pipeline name as an extra argument to the deepdive run command, different pipelines can be run very easily: No more application.conf editing.
  • Logs and outputs are placed under application root, under snapshot/.

• Adds piggy extractor that replaces the now deprecated plpy extractor.

• Includes the latest DDlog compiler with extended syntax support for writing more real world applications.

• Includes the latest Mindbender with Dashboard GUI for producing summary reports after each DeepDive run and interactively analyzing data products.

Workaround for Old Style Applications

To have an old style DeepDive applications run with this version, here's a small change you can try. You can replace your sbt or deepdive line in your run.sh script with the following deepdive env java command:

deepdive env java org.deepdive.Main -c application.conf -o output_dir

However, it is strongly recommended to upgrade to the new standard layout because more features and tools will rely on the app structure, and backward incompatible changes that break the following workaround can be introduced at any release in the future.