JSON document
From the file system
If you open a JSON document from the file system, Hackolade will detect its nature and will execute the reverse-engineering of that document. You will need to choose a database target for your model.
If you wish to include the schema for a JSON document in an existing model, the process is slightly different. With your model already opened, choose Tools > Reverse-Engineer > JSON document.
The process includes detection of ISO 8601 date/time/timestamp/duration in strings when target supports such data types.
Hackolade also supports Newline Delimited JSON as described in https://ndjson.org/
The structure of a JSON document can be imported either as an entity in the Entity Relationship Diagram, or alternatively as a model definition so it could be re-used in the model:
If you wish to force the destination of the reverse-engineering operation, you may specify the container in which the entities should be inserted.
For RDBMS targets, an additional option appears, that allows automatic normalization of complex data types:
From cloud storage and schema registries
JSON files and schemas can also be reverse-engineered from AWS S3, Azure Blob Storage/ADLS, and Google Cloud Storage.
AWS S3
Give a meaningful name to the connection to identify it for later, and provide proper URI to your S3 bucket, and optional folder path.
If the S3 bucket is private, you must also provide authentication parameters (Access key id and Secret access key):
If you wish to handle AWS authentication through the credentials file, you may leave blank the Access Key ID and Secret Access Key fields, knowing that Hackolade supplies credentials following the recommendations described here.
Azure Blob Storage and ADLS
Give a meaningful name to the connection to identify it for later, and provide proper Container name and Storage account name.
Note: be careful to only mention the storage account name, i.e. NOT a full URL or anything other than the storage account name.
If you wish to filter files, you may enter a file name prefix:
Anonymous Authentication
If the storage is public, you may choose the anonymous method:
If the storage account is private, Azure provides a choice of 3 methods with different levels of granularity depending on your security requirements:
Storage Access Key
The storage access key is similar to a root password for your storage account. Always be careful to protect your access keys. Microsoft recommends that you use Azure Key Vault to manage your access keys, and that you regularly rotate and regenerate your keys.
Select the authentication method and paste the key into the Storage Access Key field:
Shared Access Signature
A shared access signature (SAS) provides secure delegated access to resources in your storage account, with granular control over how a client can access your data: user delegation, service or account.
For Hackolade to be able to reverse-engineer, the minimum rights are as shown here:
After clicking the button to generate, copy the SAS token from the Azure portal, and paste it in the SAS Token field:
Shared Access Token per container
It is possible to generate tokens for specific containers in the "Shared access tokens" menu option of the container:
The minimum required rights for our reverse-engineering process to succeed are: read and list.
After clicking the button to generate, copy the Blob SAS token from the Azure portal, and paste it in the SAS Token field:
Google Cloud Storage
Give a meaningful name to the connection to identify it for later, and provide proper URI to your GCS bucket, and optional folder path.
If the bucket is private, you must also access to the Private key:
