datahub

ingest metadata of datasets

• ingest binary files from s3
• ingest tables(parquet, csv) from s3
• ingest custom datasets from s3
  • virtual files
  • files with special metadata contains in its content

query metadata of datasets

• query containers
• query datasets
• custom query

add extra indexes into elastic search

• add extra indexes for specific customProperties
• query datasets by extra indexes added