Ampool ADS v1.4.1 improves prior releases, by improving both functionality and performance. Moreover, the core is upgraded to the latest release of Apache Geode (v.1.1).
What's New in 1.4.1¶
With this release, a new immutable data abstraction, FTable, is introduced. Following are the highlights of this release:
Core data store
- Rebase to the latest release Geode v1.1. For a full list of changes, please refer the Geode 1.1 Release Notes
- User-level APIs have been re-factored to use less code while communicating/ integrating with Ampool ADS
Interfaces & APIs
- CLI (MASH): Added support for exporting data from the command-line. For a complete list, please refer to Table Command Reference
- REST: Added basic data management commands for FTable. Now you can append and get rows from these tables and even use filters to scan for a relevant subset of data
- Spark: With this release, both Spark 1.6/ Scala 2.10 and Spark 2.1.0/ Scala 2.11 connectors are available. This enables users to work on the same table using different Spark client versions!
- Kafka: ampool-connect-kafka can now store data from multiple kafka (topics) to both ampool table types (MTable and FTable).
Besides, several internal enhancements were made to improve performance for insert, update & scan operations, and multiple usability fixes.
Following packages are available for download from Ampool's (S3) website:
Ampool Base Package (ampool-1.4.1.tar.gz): Includes Ampool core (MTable, FTable, CoProcessors, Local DiskStore for recovery, etc) and core interfaces (MASH & Java API)
Ampool Compute/ Ingest Connectors:
Spark (ampool-1.4.1-spark_1.6.tar.gz and ampool-1.4.1-spark_2.1.tar.gz )
Installing Ampool v1.4.1¶
Core Ampool Server & Locator¶
- Untar the binaries in a new installation directory. After extracting the contents from the package, you should see the following directory structure:
bin config docs examples lib tools
- For launching ampool services, start the command-line utility MASH (Memory Analytics Shell) by typing the following from the installed ampool directory (
$ <ampool-home>/bin/mash mash>
Type 'help' for a list of commands.
For a detailed explanation of ampool services and commands, please refer to the README within the main directory.
Untar the Ampool connector packages on the Ampool client nodes (ampool-1.4.1-spark_2.1.tar.gz, ampool-hive-1.4.1.tar.gz, ampool-connect-kafka-1.4.1.tar.gz)
Refer to README in the respective packages to install and use the Ampool connectors with Spark and Hive.
Upgrading to Ampool ADS v1.4.1¶
Core Ampool Server & Locator¶
Stop the existing Ampool Server(s) and Locator(s) using Mash CLI
Untar the new Ampool core package and start the Ampool Server and Locators using new binaries.
Make sure provide the previous version's server and locator directories using --dir option in Mash CLI when starting the Server and Locators.
- Untar the newer version of connector packages and use them in place of previous version in the classpath when using Spark, Hive and Kafka with newer version of Ampool
Resolved (Major) Issues¶
|GEN-1721||Parallel data ingest through multiple kafka topics and ampool tables failed in distributed set up.|
|GEN-1618||Scan from multiple independent clients caused high CPU on server. This is now resolved.|
|GEN-1542||FTable Tier-store and its dependencies are now packaged separately.|
Known Issues & Limitations¶
|Issue Ref||Description||Workaround (if any)|
|GEN-1161||On local developer machines, if ampool services are started without specifying the host, it may bind to wifi address, which changes with moving location. In such scenarios, reconnecting to the locator from MASH fails.||Manually kill ampool (locator and server) processes and restart the services. Alternatively, specify localhost or stable network interface to bind these services.|
|Limitation||Coprocessor can not be called on an empty table.||Endpoint coprocessor execution is not supported on empty table, To check for empty table use, MTable.isEmpty() API.|
|GEN-1144||runExamples script with already running ampool cluster fails.||Run the runExamples script after stopping the ampool cluster.|
Versions & Compatibility¶
This distribution is based on Apache Geode release (1.0.0-incubating.M3). Following table summarizes the minimum versions supported for different connectors:
|Apache Spark||2.1.0, 1.6.0|
|Apache Hive||0.14.0, 1.2.1|
- Code examples: A set of code samples showcasing table and coprocessor API can be found under