Installation Steps

GlueSync SQL to NoSQL for AWS S3

Prerequisites

In order to have GlueSync working on your AWS S3 instance you will need to have:

  • an S3 bucket;

  • valid user credentials with permissions: arn:aws:iam::aws:policy/AmazonS3FullAccess;

  • IAM access key obtained through AWS IAM console;

  • IAM secret key obtained through AWS IAM console;

  • name of the region where the buckets belogs to.

Basic configuration example

This module can be customized by using a configuration file, in JSON format. The file name to use must be specified as parameter when launching the app, with the -f or --file tokens. The file should be composed by union of common configuration file (see here Installation Steps) and source specific configuration:

{
  ...
  // (source section omitted)
  ...
  "targetHost": "eu-central-1",
  "targetName": "myBucketName",
  "targetUsername": "IAMCredentialAccessKeyID",
  "targetPassword": "IAMCredentialSecret",
  ...

  "awsS3": {
    "entitiesMetadata": {
      "articles": [ "id", "article_name" ]
    },

    "allowObjectsDeletion": false
  },
}

The parameters that refer to a target are the parameters that describe the connection to the AWS S3 bucket.

  • targetHost: the name of the region where your bucket belongs to;

  • targetName: the name of the target bucket as represented inside your AWS S3 console (do not use the arn);

  • targetUsername: IAM Credential access key ID created under your AWS IAM console;

  • targetPassword: IAM Credential secret created under your AWS IAM console.

Other configuration parameters are the following:

AWS S3 specific configurations are listed under the awsS3 property:

  • timeoutSeconds (optional): defaults to 60. Number of seconds to set as timeout for operations involving communication with the destination;

  • allowObjectsDeletion (optional): defaults to true. If set to false tells GlueSync to skip any action that involves data deletion committed into the source database: GlueSync will propagate only events that involve Insert(s) and Update(s).

  • entitiesMetadata (optional): defaults to null. Key-value object that describes the binding of values per each table-fields that you would like to set as metadata values belonging to each stored object in the S3 bucket. Where key is the table (entity) name and as a value a list (array) of strings representing the column names.