v1.0.0

Introduction

Ampool 1.0 is the first packaged/ documented release of Ampool targeted towards near-app analytics - essentially for those users who are looking for analytics from data 'exhaust' of their Apps. This segment is also referred to embedded analytics.

What's New in 1.0.0

Following are few key highlights of this release:

Core data store

  • Key-Value Region: This is the default type of region supported.

  • Table: A tabular data abstraction of two types - ordered_versioned and unordered that supports typical C(R)UD operations (create, add, delete) in an HBase-style API

  • Co-processors are supported on the Tables, which allow for observing data changes or pushing data processing closer to where data is stored

  • Data recovery: Data is stored in memory and captured on disk for purpose of server recovery

  • Security: Kerberos authentication is supported at service level. This is particularly important for Big Data (Hadoop) installations for operation within kerberized clusters

Interfaces & APIs These are different ways to use the in-memory data store:

  • Java (language) API

  • REST API: This API is supported for apps to interact directly with the store for common data operations (Key-Value store only)

  • CLI (MASH): Basic operations for creating a table and putting/ getting values are supported

Connectors These are interfaces to existing data processing (computation) frameworks, most commonly used for analytical processing:

  • Spark: Two data abstractions supported
  • RDD for Key-Value Region
  • DataFrame for Table

  • Hive: Storage Handler for both native and external tables. Also, all Hive types are supported

Package Contents

Following packages are available for download from Ampool's (S3) website:

  • Ampool Base Package (ampool-1.0.0.tar.gz): Includes Ampool core (MTable, CoProcessors, Local DiskStore for recovery, etc) and core interfaces (MASH & Java API)

  • Ampool Compute Connectors:

  • Spark (ampool-spark-1.0.0.tar.gz)

  • Hive (ampool-hive-1.0.0.tar.gz)

Installing Ampool v 1.0.0

Core Server & Locator

  • Untar the binaries in a new installation directory. After extracting the contents from the package, you should see the following directory structure:
bin
config
docs
examples
lib
tools
  • For launching ampool services, start the command-line utility MASH (Memory Analytics Shell) by typing the following from the installed ampool directory (ampool-home):
$ <ampool-home>/bin/mash
mash>

Type 'help' for a list of commands.

For a detailed explanation of ampool services and commands, please refer to the README within the main directory.

Connectors

  • Untar the Ampool connector packages on the Ampool client nodes (ampool-spark-1.0.0.tar.gz, ampool-hive-1.0.0.tar.gz)

  • Refer to README in the respective packages to install and use the Ampool connectors with Spark and Hive.

Known Issues & Limitations

Issue Ref Description Workaround (if any)
- CLI (MASH) does not support scan or delete operation -
- Coprocessors are not supported at a table split level. -
- REST API is only supported for Key-Value store. -
GEN-906 Endpoint Coprocessor is not supported for UNORDERED Table type. -
GEN-974 Get fails with "java.lang.AssertionError: Member id should not be null for persistent version tags". None available, retry the get operation.
GEN-926 Occasionally, after server restart, meta data update is not complete before execution of a request for co-processor, which fails to get serverLocation for any of the buckets, hence co processor is not executed. Provide a small sleep/ wait before executing a co-processor after service restart.

Versions & Compatibility

This distribution is based on the latest Apache Geode release (1.0.0-incubating.M3). Following table summarizes the minimum versions supported for different connectors:

Connector Version
Apache Spark 1.5.1, 1.6.0
Scala 2.10
Apache Hive 0.14.0, 1.2.1
  • Java API: You can find javadoc for Ampool client API under <installation_dir>/docs/api folder.

  • Code examples: A set of code samples showcasing table and coprocessor API can be found under <installation_dir>/examples folder.