DataHub Releases

Summary

Version	Release Date	Links
v0.10.0	2023-02-07	Release Notes, View on GitHub
v0.9.6.1	2023-01-31	Release Notes, View on GitHub
v0.9.6	2023-01-13	Release Notes, View on GitHub
v0.9.5	2022-12-23	Release Notes, View on GitHub
v0.9.4	2022-12-20	View on GitHub
v0.9.3	2022-11-30	View on GitHub
v0.9.2	2022-11-04	View on GitHub
v0.9.1	2022-10-31	View on GitHub
v0.9.0	2022-10-11	View on GitHub
v0.8.45	2022-09-23	View on GitHub
v0.8.44	2022-09-01	View on GitHub
v0.8.43	2022-08-09	View on GitHub
v0.8.42	2022-08-03	View on GitHub
v0.8.41	2022-07-15	View on GitHub
v0.8.40	2022-06-30	View on GitHub
v0.8.39	2022-06-24	View on GitHub
v0.8.38	2022-06-09	View on GitHub
v0.8.37	2022-06-09	View on GitHub
v0.8.36	2022-06-02	View on GitHub
v0.8.35	2022-05-18	View on GitHub
v0.8.34	2022-05-04	View on GitHub
v0.8.33	2022-04-15	View on GitHub
v0.8.32	2022-04-04	View on GitHub
v0.8.31	2022-03-17	View on GitHub
v0.8.30	2022-03-17	View on GitHub
v0.8.29	2022-03-10	View on GitHub
v0.8.28	2022-03-07	View on GitHub
v0.8.27	2022-02-23	View on GitHub

DataHub v0.10.0

Released on 2023-02-07 by @david-leifker.

Release Highlights

Potential Downtime

This release introduces substantial improvements to search functionality which require reindexing indices.

During the reindexing:

• a system-update job will set indices to read-only and create a backup/clone of each index
• new components will be prevented from start-up until the reindex completes
• Helm deployments will go into read-only mode and new ingestion runs will fail

This process can take anywhere from 5 minutes to multiple hours; as rough estimate, please expect it to take 1 hour for every 2.3 million entities. After the reindex is complete, please check your ingestion run to re-run any that did not complete.

If you are deploying containers yourself

If you're deploying the Docker containers yourself (without Helm or Docker-Compose Quickstart), then you'll need to ensure that you first run the acryldata/datahub-upgrade docker image (v0.10.0 tag) with the following environment variables enabled.

Then, run the container this with the command

docker run acryldata/datahub-upgrade:v0.10.0 -u SystemUpdate

For the full set of environment variables required, check out the default docker.env provided for Docker Compose deployments.

This will run the required reindex against your elasticsearch instance, after which other DataHub components should start correctly. If you do not run the datahub-upgrade container successfully, other components in the stack will fail to start correctly.

User Experience

We have some really exciting improvements to the DataHub user experience in this release!

Improved documentation editor, contributed by @ngamanda and the Grab Team. This work provides a much more intuitive documentation editing experience within the UI, providing “what you see is what you get” formatting & removing the need for markdown expertise.

Additionally, you can easily:

• Add links to other entities/users within DataHub
• embed and resize tables & images
• toggle between font sizes and formats
• embed syntax-highlighted code blocks

<img src="https://user-images.githubusercontent.com/114954101/217367791-3d392ae4-f422-4188-8d3c-768cb7c120ea.png" width="800">

Filter lineage graphs based on time windows You can now easily see the full lineage graph of an entity at a specific point in time. This makes it much easier to understand how interdependencies have evolved over time and to troubleshoot data issues in the past.

Improvements in Search As noted above, we have rolled out substantial improvements to Search functionality, making it easier than ever for end-user to find the entities that matter most. This release includes:

• Stemm & Synonyms
• Search by full or partial URN
• Autocomplete improvements
• Quoted search analyzer for exact & prefix match

Metadata Ingestion

Here are some of the most notable ingestion-related improvements:

• Redshift: You can now extract lineage information from unload queries – thanks for the contrib, @mmmeeedddsss
• PowerBI: Ingestion now maps Workspaces to DataHub Containers – thanks for the contrib, @looppi
• BigQuery: You can now extract lineage metadata from the Catalog API – thanks for the crontrib, @PatrickfBraz
• Glue: Ingestion now uses table name as the human-readable name – thanks for the contrib, @danielcmessias

Developer Experience

• This release introduces DataHub Lite - a new experimental lightweight implementation of DataHub. It is intended to enable local developer tooling use-cases such as simple access to metadata for scripts and other tools. DataHub Lite is compatible with the DataHub metadata format and all the ingestion connectors that DataHub supports. Checkout the docs here.

Breaking Changes

[#7103](https://github.com/datahub-project/datahub/pull/7103) This should only impact users who have configured explicit non-default names for DataHub's Kafka topics. The environment variables used to configure Kafka topics for DataHub used in the kafka-setup docker image have been updated to be in-line with other DataHub components, for more info see our docs on Configuring Kafka in DataHub . They have been suffixed with _TOPIC where as now the correct suffix is _TOPIC_NAME. This change should not affect any user who is using default Kafka names.

What's Changed

• fix(ci): only scan on master branch by @anshbansal in https://github.com/datahub-project/datahub/pull/7047
• fix(ci): use trivy offline scanning by @anshbansal in https://github.com/datahub-project/datahub/pull/7050
• docs(get-started) Simplify copy on Get Started landing page by @maggiehays in https://github.com/datahub-project/datahub/pull/7043
• fix(ingest/kafka): fix ResourceType import error for confluent_kafka<1.9.0 by @mayurinehate in https://github.com/datahub-project/datahub/pull/7046
• docs(dbt): fix indentation in dbt meta mapping docs by @jx2lee in https://github.com/datahub-project/datahub/pull/7045
• fix(ingest): temporarily disable vertica tests by @hsheth2 in https://github.com/datahub-project/datahub/pull/7059
• feat(editor): improve documentation editor using Remirror by @ngamanda in https://github.com/datahub-project/datahub/pull/6631
• fix(bootstrap): add EDIT_LINEAGE privilege to some default policies by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7060
• feat(ingest): add entity registry in codegen by @hsheth2 in https://github.com/datahub-project/datahub/pull/6984
• feat(ingest): extract powerbi endorsements to tags by @looppi in https://github.com/datahub-project/datahub/pull/6638
• feat(ingestion): pull metabase database, schema names from raw query and api by @remisalmon in https://github.com/datahub-project/datahub/pull/7039
• fix(ingest): support multiple entity_registry sections by @hsheth2 in https://github.com/datahub-project/datahub/pull/7066
• ci(ingest): add flag to skip tests but run codegen during release by @hsheth2 in https://github.com/datahub-project/datahub/pull/7067
• fix(ingest): preserve dbt column name casing by @hsheth2 in https://github.com/datahub-project/datahub/pull/7063
• fix(ingest/tableau): fix node limit exceeded error for workbooks query by @mayurinehate in https://github.com/datahub-project/datahub/pull/7068
• fix(build/airflow): Fixing gradlew path by @treff7es in https://github.com/datahub-project/datahub/pull/7069
• feat(ingest): support snapshots in dbt and dbt-cloud by @hsheth2 in https://github.com/datahub-project/datahub/pull/7062
• fix(ui) Fix duplicate schema field rendering with siblings by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7057
• refactor(ingest/athena): Replace s3_staging_dir parameter in Athena source with query_result_location by @bossenti in https://github.com/datahub-project/datahub/pull/7044
• feat(ingest): fix handling of unions with aliases in post restli conversion by @hsheth2 in https://github.com/datahub-project/datahub/pull/7058
• fix(ui) Make checkboxes in ingestion forms easier to see by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7061
• fix(ingest): support git clone of non-github repos by @hsheth2 in https://github.com/datahub-project/datahub/pull/7065
• feat(ingest): reporting revamp, part 1 by @hsheth2 in https://github.com/datahub-project/datahub/pull/7031
• fix(secret-service): fix default encrypt key by @david-leifker in https://github.com/datahub-project/datahub/pull/7074
• feat(datahub-lite): introduces a new experimental lightweight impleme… by @shirshanka in https://github.com/datahub-project/datahub/pull/7052
• feat(datahub-lite): adding tab completion, small serialization fixes by @shirshanka in https://github.com/datahub-project/datahub/pull/7079
• docs: add docs for managed DataHub v0.1.72 by @anshbansal in https://github.com/datahub-project/datahub/pull/7070
• docs(readme): add inovex as adopter by @DSchmidtDev in https://github.com/datahub-project/datahub/pull/7077
• docs: add warning about clearing cookies for login by @anshbansal in https://github.com/datahub-project/datahub/pull/7084
• feat(cache): add hazelcast distributed cache option by @RyanHolstien in https://github.com/datahub-project/datahub/pull/6645
• docs(datahub-lite): small improvement for zsh tab completion by @shirshanka in https://github.com/datahub-project/datahub/pull/7085
• fix(ingest/bigquery): clear stateful ingestion correctly by @hsheth2 in https://github.com/datahub-project/datahub/pull/7075
• fix(graphql): Return with appropriate status code instead of stacktrace by @szalai1 in https://github.com/datahub-project/datahub/pull/7086
• fix(sso): Clear cookies on SSO redirect error by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7088
• fix(docs): add missing mutation literal by @ruedigerblock in https://github.com/datahub-project/datahub/pull/7082
• fix(ui): display the correct access token expiry in AccessTokenModal by @ngamanda in https://github.com/datahub-project/datahub/pull/7078
• fix(cli/lite): fix datahub lite serve command by @hsheth2 in https://github.com/datahub-project/datahub/pull/7089
• fix(profiling): Fix syntax for APPROX_COUNT_DISTINCT on bigquery and snowflake by @feljen in https://github.com/datahub-project/datahub/pull/7087
• fix(ingest): fix logic error of google protobuf wrapper type. by @wngus606 in https://github.com/datahub-project/datahub/pull/7076
• feat(ui): Documentation Editor Improvements by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7072
• fix(uri): marks uri field as deprecated, removes problem code, and adds coercer for usages of URI typeref by @RyanHolstien in https://github.com/datahub-project/datahub/pull/7093
• fix(build): postgres docker secret by @david-leifker in https://github.com/datahub-project/datahub/pull/7092
• fix(ingest/snowflake): handle corrupted snowflake OCSP cache file by @hsheth2 in https://github.com/datahub-project/datahub/pull/7095
• refactor(ingest): Refactoring container creation to common place by @treff7es in https://github.com/datahub-project/datahub/pull/6877
• feat(ingest): move datahub-lite to optional dep and add shim when missing by @hsheth2 in https://github.com/datahub-project/datahub/pull/7097
• fix(docker): support non amd64 dockerize in setup containers by @tonycsoka in https://github.com/datahub-project/datahub/pull/7091
• test(ingest): fix kafka admin client mocking by @hsheth2 in https://github.com/datahub-project/datahub/pull/7098
• fix(build): Fix postgres setup gha by @david-leifker in https://github.com/datahub-project/datahub/pull/7104
• fix(ingest/profile): properly quoting approx_count_distinct by @treff7es in https://github.com/datahub-project/datahub/pull/7101
• style(models): Replaces non-ASCII charactes in pdl files with ASCII c… by @nmbryant in https://github.com/datahub-project/datahub/pull/7105
• feat(ingest): hide cartesian product warnings in GE profiler by @hsheth2 in https://github.com/datahub-project/datahub/pull/7096
• feat(ingest): add removing partition pattern in spark lineage by @ssilb4 in https://github.com/datahub-project/datahub/pull/6605
• feat(redshift): Fetch lineage from unload queries by @mmmeeedddsss in https://github.com/datahub-project/datahub/pull/7041
• fix(ci): do not confirm on force for deletion by @anshbansal in https://github.com/datahub-project/datahub/pull/7106
• fix(analytics): add missing usage events causing warning in logs by @anshbansal in https://github.com/datahub-project/datahub/pull/7109
• feat(quickstart): Remove kafka-setup as a hard deployment requirement by @pedro93 in https://github.com/datahub-project/datahub/pull/7073
• fix(tests): Fixing add_users smoke test by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7116
• chore(deps): bump ua-parser-js from 0.7.32 to 0.7.33 in /docs-website by @dependabot in https://github.com/datahub-project/datahub/pull/7122
• docs(gms): clarify behavior of soft deletion in UI by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7117
• fix(kafka-setup): Make topic name consistent with other images by @pedro93 in https://github.com/datahub-project/datahub/pull/7103
• chore(deps): bump ua-parser-js from 0.7.32 to 0.7.33 in /datahub-web-react by @dependabot in https://github.com/datahub-project/datahub/pull/7123
• feat(ingest): powerbi # add powerbi workspaces to containers by @looppi in https://github.com/datahub-project/datahub/pull/6532
• fix(diffMode): prevent misconfiguration of diff mode by @RyanHolstien in https://github.com/datahub-project/datahub/pull/7127
• fix(ui) Display glossary term name in analytics page properly by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7128
• fix(ui): only use visible and enabled tabs for selected tab and routing in entity profiles by @Masterchen09 in https://github.com/datahub-project/datahub/pull/6629
• fix(htrace): remove htrace jar by @szalai1 in https://github.com/datahub-project/datahub/pull/7126
• feat(datahub-lite): simplify get response by @shirshanka in https://github.com/datahub-project/datahub/pull/7131
• fix(doc/biquery): Updating bigquery capability doc by @treff7es in https://github.com/datahub-project/datahub/pull/7136
• fix(ci): do not fail fast for matrix runs by @anshbansal in https://github.com/datahub-project/datahub/pull/7132
• refactor(ui): refactor capitalization of platform name and sub types by @Masterchen09 in https://github.com/datahub-project/datahub/pull/7099
• refactor(cli): extract method, change wording by @anshbansal in https://github.com/datahub-project/datahub/pull/7134
• docs(lineage): Updating Lineage feature guide by @maggiehays in https://github.com/datahub-project/datahub/pull/6257
• removing WIP by @laulpogan in https://github.com/datahub-project/datahub/pull/7140
• docs(oidc): Updating + improving docs around OIDC configuration by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7141
• fix(ingest): add message proto check by @tinolyu in https://github.com/datahub-project/datahub/pull/7130
• fix(ingest): use snowflake median function in profiling by @hsheth2 in https://github.com/datahub-project/datahub/pull/6987
• feat(ui): allow removing parentNodes of Glossary Nodes and Glossary Terms by @ngamanda in https://github.com/datahub-project/datahub/pull/7135
• feat(ui) Add new embedded profile to be displayed in extension by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7113
• feat(ingest): add --log-file option and show CLI logs in UI report by @hsheth2 in https://github.com/datahub-project/datahub/pull/7118
• fix(misc): NPE and GraphQL case fixes by @david-leifker in https://github.com/datahub-project/datahub/pull/7149
• fix(ingest/snowflake): fix regression in approx count distinct by @hsheth2 in https://github.com/datahub-project/datahub/pull/7146
• [docs] fix typo / add missing line for docker compose / attach overwriting system action config for confluent. by @kdongho in https://github.com/datahub-project/datahub/pull/7142
• reordering sidebar and adding homepage to apis by @laulpogan in https://github.com/datahub-project/datahub/pull/7139
• fix(ingestion): powerbi # Not all arguments converted to string by @mohdsiddique in https://github.com/datahub-project/datahub/pull/7157
• fix(ui): Sort top users by their query count in datasets stats tab by @jaykadambi in https://github.com/datahub-project/datahub/pull/7148
• refactor(ui): Updates to Manual Lineage search by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7151
• feat(ui) Build entity doesn't exist page for entity profiles by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7150
• ci(ingest): fix broken CI workflow for metadata-ingestion by @hsheth2 in https://github.com/datahub-project/datahub/pull/7161
• fix(ingest): azuread group mapping do not stop ingestion by @anshbansal in https://github.com/datahub-project/datahub/pull/7169
• fix(docs): Fixes links to docs templates by @viniciusdsmello in https://github.com/datahub-project/datahub/pull/7171
• refactor(ui ingest): Allow enabling / disabling ingestion schedule easily by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7162
• fix(ingest): switch various sources to auto_stale_entity_removal helper by @hsheth2 in https://github.com/datahub-project/datahub/pull/7158
• docs(townhall) Update Townhall History doc by @maggiehays in https://github.com/datahub-project/datahub/pull/7180
• test(ingest/delta-lake): fix spurious directory creation by @hsheth2 in https://github.com/datahub-project/datahub/pull/7179
• feat: add a linter for github actions workflows by @hsheth2 in https://github.com/datahub-project/datahub/pull/7178
• fix(quickstart): adding back kafka-setup by @szalai1 in https://github.com/datahub-project/datahub/pull/7181
• fix(docs) Fix broken links in ingestion docs by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7183
• fix(ingest/GX): fix snowflake urn generated from connection string by @mayurinehate in https://github.com/datahub-project/datahub/pull/7173
• feat(ingest): switch dbt to use auto_stale_entity_removal by @hsheth2 in https://github.com/datahub-project/datahub/pull/7160
• fix(ingest): fix issue in glue tests by @hsheth2 in https://github.com/datahub-project/datahub/pull/7185
• fix(log): logging timestamp in ISO8601 format instead of time by @anshbansal in https://github.com/datahub-project/datahub/pull/7188
• feat(ingest): bigquery - extracts lineage metadata from catalog api by @PatrickfBraz in https://github.com/datahub-project/datahub/pull/7137
• fix(ingest/tableau): show warning about token expiry for PATs by @hsheth2 in https://github.com/datahub-project/datahub/pull/7187
• fix(ingest/vertica): Fixing missing container properties by @treff7es in https://github.com/datahub-project/datahub/pull/7197
• chore(deps): bump Netty from 4.1.85.Final to 4.1.86.Final by @janhicken in https://github.com/datahub-project/datahub/pull/7191
• docs(ingestion): powerbi # Add permission for DAX and mashup expressions by @mohdsiddique in https://github.com/datahub-project/datahub/pull/7195
• feat(elasticsearch): Elasticsearch improvements by @david-leifker in https://github.com/datahub-project/datahub/pull/6894
• fix(test): spark-lineage # build task as dependency of integrationTest by @mohdsiddique in https://github.com/datahub-project/datahub/pull/7189
• chore(sample): add status removed aspect for sample data by @anshbansal in https://github.com/datahub-project/datahub/pull/7203
• docs(managed datahub): release notes for v0.1.73 by @anshbansal in https://github.com/datahub-project/datahub/pull/7194
• fix(bootstrapdata): update timestamp to be in the last 1 year by @szalai1 in https://github.com/datahub-project/datahub/pull/7206
• fix(ingest/bigquery): quoting for APPROX_COUNT_DISTINCT in BigQuery by @mryorik in https://github.com/datahub-project/datahub/pull/7207
• fix(versioning): Ensure that CLI version is always dot-delimited even in minor release versions by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7200
• fix(test): missing variables in test causing error in logs by @anshbansal in https://github.com/datahub-project/datahub/pull/7210
• feat(mlModel): mark downstream jobs as ml model downstreams lineage by @mayurinehate in https://github.com/datahub-project/datahub/pull/7205
• ci(): fix datahub-upgrade quickstart regression by @hsheth2 in https://github.com/datahub-project/datahub/pull/7217
• feat(ingest): Add custom properties to the ldap ingestion by @bda618 in https://github.com/datahub-project/datahub/pull/7125
• fix(ingest): upgrade feast to avoid build issues by @hsheth2 in https://github.com/datahub-project/datahub/pull/7218
• fix(ui) Increase the number of assertions that we query for in tab by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7215
• fix(ci): trivy code scanning fix by @anshbansal in https://github.com/datahub-project/datahub/pull/7232
• feat(glue): Use table name as human-readable name for Glue ingestion by @danielcmessias in https://github.com/datahub-project/datahub/pull/7213
• feat(ui): Supporting display of columns and storage count in previews by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7198
• fix(gms): Fixes delete references for single relationship aspects by @pedro93 in https://github.com/datahub-project/datahub/pull/7211
• docs(ingest/lineage): clarify name field in entity config for file based lineage by @mayurinehate in https://github.com/datahub-project/datahub/pull/7225
• fix(ui): typo 'Documenataion' by @vojtechneradatos in https://github.com/datahub-project/datahub/pull/7227
• fix(cli/delete): skip references prompt if deleting an aspect by @hsheth2 in https://github.com/datahub-project/datahub/pull/7220
• fix(ingest/tableau): implement workbook_page_size parameter by @hsheth2 in https://github.com/datahub-project/datahub/pull/7216
• fix(gms): Corrects MCP generation in async mode by @pedro93 in https://github.com/datahub-project/datahub/pull/7214
• fix(ingest): redshift # build late binding view lineage when sql written in upper case by @looppi in https://github.com/datahub-project/datahub/pull/7223
• fix(siblings) Fix editing of schema fields for siblings with unequal schemas by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7199
• fix(ingest-idp): emit empty GroupMembership when there are no groups by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7196
• feat(lineage): add time filtering for lineage edges by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7159
• chore(deps): bump http-cache-semantics from 4.1.0 to 4.1.1 in /docs-website by @dependabot in https://github.com/datahub-project/datahub/pull/7230
• refactor(docs): Minor language updates for kafka source doc header by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7237
• docs(website): fix feature availability dark mode styles by @jeffmerrick in https://github.com/datahub-project/datahub/pull/7233
• chore(log/docs): improve error log, docs by @anshbansal in https://github.com/datahub-project/datahub/pull/7239
• fix(dev.sh): Add context to kafka-setup build by @szalai1 in https://github.com/datahub-project/datahub/pull/7234
• feat(cli): improve docker quickstart by @hsheth2 in https://github.com/datahub-project/datahub/pull/7184
• fix(elasticsearch): fix orphan index clean up pattern, consistent top… by @david-leifker in https://github.com/datahub-project/datahub/pull/7242
• chore(deps): bump http-cache-semantics from 4.1.0 to 4.1.1 in /datahub-web-react by @dependabot in https://github.com/datahub-project/datahub/pull/7231
• Update data_platforms.json by @RainerGa in https://github.com/datahub-project/datahub/pull/7244
• fix(autocomplete): Use normal properties name instead of urn name in autocomplete by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7236
• fix(frontend logs): Silencing harmless log messages (and adding path for future) by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7254
• fix(docker): fix ability to use non-default reg by @david-leifker in https://github.com/datahub-project/datahub/pull/7250
• logging(elasticsearch): improve messaging in orphan index detection by @david-leifker in https://github.com/datahub-project/datahub/pull/7246
• chore(ci): update base image dependencies by @anshbansal in https://github.com/datahub-project/datahub/pull/7248
• docs(graphql): remove reference of non-existent gms.graphql by @mayurinehate in https://github.com/datahub-project/datahub/pull/7240
• Add graphql error and call metrics at startuptime by @szalai1 in https://github.com/datahub-project/datahub/pull/7226
• docs(ingest): update kafka connect doc, simplify starter recipe by @mayurinehate in https://github.com/datahub-project/datahub/pull/7243
• fix(cli): update message when pulling docker images by @mayurinehate in https://github.com/datahub-project/datahub/pull/7241
• fix(ingest/tableau): handle missing query in tableau views by @hsheth2 in https://github.com/datahub-project/datahub/pull/7186
• feat(ingest/s3): use latest file to infer schema metadata by @mayurinehate in https://github.com/datahub-project/datahub/pull/7202
• fix(schema-blame): check if list of ChangeTransactions is empty before processing by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7263
• fix(change-events): guard against NPE's by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7264
• fix(docker): add env variable to control mysql setup image, sort dock… by @shirshanka in https://github.com/datahub-project/datahub/pull/7266
• chore(logs): clean logs scanning location by @anshbansal in https://github.com/datahub-project/datahub/pull/7261
• fix(profile): use department name if available by @anshbansal in https://github.com/datahub-project/datahub/pull/7257
• fix(async ingest): Fix async ingest path by @pedro93 in https://github.com/datahub-project/datahub/pull/7269
• fix(compose): fix override file missing container by @david-leifker in https://github.com/datahub-project/datahub/pull/7270
• fix(ui): fix spacing on share buttons by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7272

New Contributors

• @bossenti made their first contribution in https://github.com/datahub-project/datahub/pull/7044
• @ruedigerblock made their first contribution in https://github.com/datahub-project/datahub/pull/7082
• @feljen made their first contribution in https://github.com/datahub-project/datahub/pull/7087
• @tonycsoka made their first contribution in https://github.com/datahub-project/datahub/pull/7091
• @tinolyu made their first contribution in https://github.com/datahub-project/datahub/pull/7130
• @kdongho made their first contribution in https://github.com/datahub-project/datahub/pull/7142
• @jaykadambi made their first contribution in https://github.com/datahub-project/datahub/pull/7148
• @viniciusdsmello made their first contribution in https://github.com/datahub-project/datahub/pull/7171
• @mryorik made their first contribution in https://github.com/datahub-project/datahub/pull/7207
• @danielcmessias made their first contribution in https://github.com/datahub-project/datahub/pull/7213
• @vojtechneradatos made their first contribution in https://github.com/datahub-project/datahub/pull/7227
• @RainerGa made their first contribution in https://github.com/datahub-project/datahub/pull/7244

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.9.6...v0.10.0

DataHub v0.9.6.1

Released on 2023-01-31 by @david-leifker.

Release Highlights

Please upgrade from 0.9.6 ASAP to avoid ongoing issues creating and using secrets.

Important Release Notes

With this release, if you are using Neo4J as your graph implementation, you need to set: GRAPH_SERVICE_DIFF_MODE_ENABLED=false

For GMS (or MAE Consumer for standalone mode).

Bug fix for secrets encryption

• Prevents decryption errors for existing secrets
• Affects reading ingestion secret created with a previous release
• Affects native user password validation

What's Changed

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.9.6...v0.9.6.1

DataHub v0.9.6

Released on 2023-01-13 by @maggiehays.

---

⚠️ This Release has been patched. Please upgrade to 0.9.6.1 ⚠️

As of January 19th, 2023 0.9.6.1 is now the official release build, and should be used over 0.9.6. Upgrade to 0.9.6.1 when possible to avoid issues creating and using secrets.

</br></br>

Release Highlights

Important Release Notes

With this release, if you are using Neo4J as your graph implementation, you need to set: GRAPH_SERVICE_DIFF_MODE_ENABLED=false

For GMS (or MAE Consumer for standalone mode).

User Experience

• We now support embedding Dashboards, Charts, and Datasets. This allows us to do things like directly embed Looker / Tableau / Mode / Redash Looks, Dashboards, Explores into the Dataset pages themselves.

• [Experimental] You can now customize the number of queries displayed on the Query tab of a Dataset entity

• Improved error messaging for bulk editing via the UI

Metadata Ingestion

• Update to data profiling to allow configurable number of sample values to be returned
• Postgres ingestion now supports emitting lineage edges for Views - shoutout to @LucasRoesler for the contribution!
• Snowflake ingestion now supports extracting tags - shoutout to @frsann for the contribution!
• Vertica ingestion now supports projections and lineage- thanks for the contribution, @vishalkSimplify!
• Glue ingestion now emits an s3 lineage edge when data was written with an s3a/s3n client - thanks for the contribution, @danielli-ziprecruiter!

Developer Experience

• Fixes quickstart/docker compose issues for M1 machines
• Improvements in reliability and performance of the Restli Service endpoints for ingestion:
  • Scale Restli Service thread pool based on CPU
  • Add retry (exp backoff) to Restli Entity Client
  • MCE no longer relies on GMS for Restli service
  • Converted Restli Service from standalone servlet to Spring injectable
  • Docker build externalized (significantly faster on m1, <7 minute build times, based on this)
  • Frontend asset generation refactor (causing tests to fail intermittently)

What's Changed

• feat(ingest): add pydantic helper for removed fields by @hsheth2 in https://github.com/datahub-project/datahub/pull/6853
• chore(0.9.5): Bump defaults for release v0.9.5 by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6856
• Revert "fix(ci): remove warnings due to deprecated action" by @anshbansal in https://github.com/datahub-project/datahub/pull/6857
• refactor(restli-mce-consumer) by @david-leifker in https://github.com/datahub-project/datahub/pull/6744
• fix(ci): reduce smoke test run time by @anshbansal in https://github.com/datahub-project/datahub/pull/6841
• fix(security): require signed/encrypted jwt tokens by @david-leifker in https://github.com/datahub-project/datahub/pull/6565
• feat(ingest): update profiling to fetch configurable number of sample values by @mayurinehate in https://github.com/datahub-project/datahub/pull/6859
• feat(ingest/airflow): support raw dataset urns in airflow lineage by @hsheth2 in https://github.com/datahub-project/datahub/pull/6854
• refactor(graphql): make graphqlengine easier to use by @anshbansal in https://github.com/datahub-project/datahub/pull/6865
• fix(kafka): datahub-upgrade job by @david-leifker in https://github.com/datahub-project/datahub/pull/6864
• feat(ingest): pass timeout config in kafka admin client api calls by @mayurinehate in https://github.com/datahub-project/datahub/pull/6863
• chore(ingest): loosen requirements file by @hsheth2 in https://github.com/datahub-project/datahub/pull/6867
• feat(ingest): upgrade pydantic version by @cccs-eric in https://github.com/datahub-project/datahub/pull/6858
• fix(elasticsearch): fixes out of order runId writes by @david-leifker in https://github.com/datahub-project/datahub/pull/6845
• chore(ingest): loosen additional requirements by @hsheth2 in https://github.com/datahub-project/datahub/pull/6868
• feat(ingest): bigquery/snowflake - Store last profile date in state by @treff7es in https://github.com/datahub-project/datahub/pull/6832
• docs(google-analytics): Correct grammatical error in README.md by @jx2lee in https://github.com/datahub-project/datahub/pull/6870
• feat(CI): add venv caching by @szalai1 in https://github.com/datahub-project/datahub/pull/6843
• feat(ingest/snowflake): handle failures gracefully and raise permission failures by @mayurinehate in https://github.com/datahub-project/datahub/pull/6748
• fix(runid): always update runid, except when queued by @david-leifker in https://github.com/datahub-project/datahub/pull/6876
• fix(ingest): conditionally include env in assertion guid by @hsheth2 in https://github.com/datahub-project/datahub/pull/6811
• chore(ci): update dependencies docs-website by @anshbansal in https://github.com/datahub-project/datahub/pull/6871
• feat(ui) - Add a custom error message for bulk edit to add clarity by @mkamalas in https://github.com/datahub-project/datahub/pull/6775
• docs(adding users): Refreshing the docs for adding new DataHub Users by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6879
• test(mce-consumer): mockbeans by @david-leifker in https://github.com/datahub-project/datahub/pull/6878
• feat(ingest): avoid embedding serialized json in metadata files by @hsheth2 in https://github.com/datahub-project/datahub/pull/6742
• refactor(gradle): move the local docker registry to common location by @david-leifker in https://github.com/datahub-project/datahub/pull/6881
• refactor(smoke): use env variables by @anshbansal in https://github.com/datahub-project/datahub/pull/6866
• fix(lint): pin pydantic version by @anshbansal in https://github.com/datahub-project/datahub/pull/6886
• refactor(docs): Correctly spell elasticsearch in docs by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6880
• fix(ingest): okta undefined variable error by @anshbansal in https://github.com/datahub-project/datahub/pull/6882
• fix(ci): reduce flakiness in add_users, siblings smoke test by @anshbansal in https://github.com/datahub-project/datahub/pull/6883
• fix(ingest): fall back to default table comment method for all Trino query errors by @marvin-roesch in https://github.com/datahub-project/datahub/pull/6873
• test(misc): misc test updates by @david-leifker in https://github.com/datahub-project/datahub/pull/6890
• deprecate(ingest): bigquery - Removing bigquery-legacy source by @treff7es in https://github.com/datahub-project/datahub/pull/6851
• chore(ingest): remove inferred args to MCPW, part 1 by @hsheth2 in https://github.com/datahub-project/datahub/pull/6819
• test(ingest/kafka-connect): make docker setup more reliable by @hsheth2 in https://github.com/datahub-project/datahub/pull/6902
• fix(ingest): profiling (bigquery) - Address biquery profiling query error due to timestamp vs data mismatch by @treff7es in https://github.com/datahub-project/datahub/pull/6874
• fix(cli): Make datahub quickstart work with latest docker compose in M1 by @pedro93 in https://github.com/datahub-project/datahub/pull/6891
• fix(cli): fix delete urn cli bug + stricter type annotations by @hsheth2 in https://github.com/datahub-project/datahub/pull/6903
• fix(ingest/airflow): reorder imports to avoid cyclical dependencies by @stijndehaes in https://github.com/datahub-project/datahub/pull/6719
• feat: remove jq requirement + tweak modeldocgen args by @hsheth2 in https://github.com/datahub-project/datahub/pull/6904
• chore(ingest): loosen pyspark and pydeequ deps by @hsheth2 in https://github.com/datahub-project/datahub/pull/6908
• docs(ingest/looker): fix typos + update lookml github action example by @hsheth2 in https://github.com/datahub-project/datahub/pull/6910
• fix(ingest/metabase): use card_id in dashboard to chart lineage by @ccpypy in https://github.com/datahub-project/datahub/pull/6583
• fix(es-setup): create data stream on non-aws by @szalai1 in https://github.com/datahub-project/datahub/pull/6926
• Adding missing Platform logos by @maggiehays in https://github.com/datahub-project/datahub/pull/6892
• feat(ingestion): PowerBI# Improve PowerBI source ingestion by @mohdsiddique in https://github.com/datahub-project/datahub/pull/6549
• Fix compose context for kafka-setup by @szalai1 in https://github.com/datahub-project/datahub/pull/6923
• feat(backend): Supporting Embeddable Previews for Dashboards, Charts, Datasets by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6875
• chore(deps): bump json5 from 2.2.1 to 2.2.3 in /docs-website by @dependabot in https://github.com/datahub-project/datahub/pull/6930
• chore(deps): bump json5 from 1.0.1 to 1.0.2 in /datahub-web-react by @dependabot in https://github.com/datahub-project/datahub/pull/6931
• fix(ci): managed ingestion test fix by @anshbansal in https://github.com/datahub-project/datahub/pull/6946
• feat(ingest): add include_table_location_lineage flag for SQL common by @hsheth2 in https://github.com/datahub-project/datahub/pull/6934
• feat(ingest): allow extracting snowflake tags by @frsann in https://github.com/datahub-project/datahub/pull/6500
• chore(ingest): unpin pydantic dep by @hsheth2 in https://github.com/datahub-project/datahub/pull/6909
• chore(ingest): partially revert pyspark dep from #6908 by @hsheth2 in https://github.com/datahub-project/datahub/pull/6954
• fix(ingest): use branch info when cloning git repos by @hsheth2 in https://github.com/datahub-project/datahub/pull/6937
• chore(ingest): remove inferred args to MCPW, part 2 by @hsheth2 in https://github.com/datahub-project/datahub/pull/6905
• fix(ingest/unity): simplify MCP generation and reporting by @hsheth2 in https://github.com/datahub-project/datahub/pull/6911
• chore(ci): parallelise build and test workflow to reduce time by @anshbansal in https://github.com/datahub-project/datahub/pull/6949
• fix(frontend): sasl.client.callback.handler.class by @szalai1 in https://github.com/datahub-project/datahub/pull/6962
• chore(react): remove outdated cypress tests and dependency by @anshbansal in https://github.com/datahub-project/datahub/pull/6948
• fix(ci): restrict GE to fix build issues by @anshbansal in https://github.com/datahub-project/datahub/pull/6967
• feat(queries): [Experimental] Allow customization of # of queries in Query tab via env var by @gabe-lyons in https://github.com/datahub-project/datahub/pull/6964
• feat(ingest/postgres): emit lineage for postgres views by @LucasRoesler in https://github.com/datahub-project/datahub/pull/6953
• feat(ingest/vertica): support projections and lineage in vertica by @vishalkSimplify in https://github.com/datahub-project/datahub/pull/6785
• fix(ingest): add missing dep for powerbi by @hsheth2 in https://github.com/datahub-project/datahub/pull/6969
• Docs fixes week of 12 22 by @laulpogan in https://github.com/datahub-project/datahub/pull/6963
• fix(ingest): unfreeze bigquery/snowflake column dataclass by @mayurinehate in https://github.com/datahub-project/datahub/pull/6921
• chore(frontend) Remove unused dependencies from package.json by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6974
• chore: misc fixes by @anshbansal in https://github.com/datahub-project/datahub/pull/6966
• feat(ingest/glue): emit s3 lineage for s3a and s3n schemes by @danielli-ziprecruiter in https://github.com/datahub-project/datahub/pull/6788
• fix(kafka-setup): Make kafka-setup run with multiple threads by @pedro93 in https://github.com/datahub-project/datahub/pull/6970
• feat(ingest): mark database_alias and env as deprecated by @hsheth2 in https://github.com/datahub-project/datahub/pull/6901
• fix(docs): Updating Tag, Glossary Term docs to point to correct GraphQL methods by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6965
• chore(deps): bump certifi from 2020.12.5 to 2022.12.7 in /metadata-ingestion/src/datahub/ingestion/source/feast_image by @dependabot in https://github.com/datahub-project/datahub/pull/6979
• fix(ingest): profiling - Fixing issue with the wrong timestamp stored in check by @treff7es in https://github.com/datahub-project/datahub/pull/6978
• config(quickstart): enable auto-reindex for quickstart by @david-leifker in https://github.com/datahub-project/datahub/pull/6983
• feat(privileges) - Create a privilege to manage glossary children recursively by @mkamalas in https://github.com/datahub-project/datahub/pull/6731
• chore(ingest): finish removing feast-legacy by @hsheth2 in https://github.com/datahub-project/datahub/pull/6985
• feat(ingest): add import descriptions of two or more nested messages by @wngus606 in https://github.com/datahub-project/datahub/pull/6959
• feat(docs) Add feature guide for Manual Lineage by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6933
• docs(rfc): Serialising GMS Updates with Preconditions by @mattmatravers in https://github.com/datahub-project/datahub/pull/5818
• fix(ingest/kafka-connect) support newer version of debezium by @jaegwonseo in https://github.com/datahub-project/datahub/pull/6943
• fix(docs): build and broken snowflake docs fix by @anshbansal in https://github.com/datahub-project/datahub/pull/6997
• fix(ingest): bigquery - views in case more than 1 datasets with views by @anshbansal in https://github.com/datahub-project/datahub/pull/6995
• fix(docs): Renaming Business Glossary Doc by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7001
• fix(ingest/snowflake): fix type annotations + refactor get_connect_args by @hsheth2 in https://github.com/datahub-project/datahub/pull/7004
• fix(docs): Changing the platform event topic name in kafka custom topic docs by @blankon123 in https://github.com/datahub-project/datahub/pull/7007
• fix(docs): fix name of privilege referenced in posts doc by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7002
• fix(SSO): Correctly redirect to originally requested URL in SSO by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7011
• fix(ingest): remove dead code from tests by @hsheth2 in https://github.com/datahub-project/datahub/pull/7005
• feat(ingestion): Tableau # Embed links by @mohdsiddique in https://github.com/datahub-project/datahub/pull/6994
• feat(auth) Update auth cookies to have same-site none for chrome extension by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6976
• docs(website): DPG WIP by @maggiehays in https://github.com/datahub-project/datahub/pull/6998
• docs: resize datahub logo by @hsheth2 in https://github.com/datahub-project/datahub/pull/7014
• fix(kafka-setup): Remove reference to non-existing topic by @pedro93 in https://github.com/datahub-project/datahub/pull/7019
• fix(ingest): powerbi # use display name field as title for powerbi report page by @looppi in https://github.com/datahub-project/datahub/pull/7017
• feat(auth) Allow session ttl to be configurable by env variable by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7022
• fix(ui): URL Encode all Entity Profile URLs by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7023
• fix(ui ingest): Fix test connection when stateful ingest is enabled by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7013
• docs(sso) move root user warning to earlier in SSO guides by @maggiehays in https://github.com/datahub-project/datahub/pull/7028
• fix(ingest/looker): add clarity in chart input parsing logs by @hsheth2 in https://github.com/datahub-project/datahub/pull/7003
• chore(ingest): remove duplicate data_platform.json file by @hsheth2 in https://github.com/datahub-project/datahub/pull/7026
• feat(ingestion): PowerBI # Remove corpUserInfo aspect ingestion by @mohdsiddique in https://github.com/datahub-project/datahub/pull/7034
• fix(metadata-models): remove unnecessary bin folder by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7035
• fixing typos by @maggiehays in https://github.com/datahub-project/datahub/pull/7030

New Contributors

• @marvin-roesch made their first contribution in https://github.com/datahub-project/datahub/pull/6873
• @stijndehaes made their first contribution in https://github.com/datahub-project/datahub/pull/6719
• @ccpypy made their first contribution in https://github.com/datahub-project/datahub/pull/6583
• @LucasRoesler made their first contribution in https://github.com/datahub-project/datahub/pull/6953
• @vishalkSimplify made their first contribution in https://github.com/datahub-project/datahub/pull/6785
• @wngus606 made their first contribution in https://github.com/datahub-project/datahub/pull/6959
• @jaegwonseo made their first contribution in https://github.com/datahub-project/datahub/pull/6943
• @blankon123 made their first contribution in https://github.com/datahub-project/datahub/pull/7007

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.9.5...v0.9.6

What's Changed

• feat(ingest): add pydantic helper for removed fields by @hsheth2 in https://github.com/datahub-project/datahub/pull/6853
• chore(0.9.5): Bump defaults for release v0.9.5 by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6856
• Revert "fix(ci): remove warnings due to deprecated action" by @anshbansal in https://github.com/datahub-project/datahub/pull/6857
• refactor(restli-mce-consumer) by @david-leifker in https://github.com/datahub-project/datahub/pull/6744
• fix(ci): reduce smoke test run time by @anshbansal in https://github.com/datahub-project/datahub/pull/6841
• fix(security): require signed/encrypted jwt tokens by @david-leifker in https://github.com/datahub-project/datahub/pull/6565
• feat(ingest): update profiling to fetch configurable number of sample values by @mayurinehate in https://github.com/datahub-project/datahub/pull/6859
• feat(ingest/airflow): support raw dataset urns in airflow lineage by @hsheth2 in https://github.com/datahub-project/datahub/pull/6854
• refactor(graphql): make graphqlengine easier to use by @anshbansal in https://github.com/datahub-project/datahub/pull/6865
• fix(kafka): datahub-upgrade job by @david-leifker in https://github.com/datahub-project/datahub/pull/6864
• feat(ingest): pass timeout config in kafka admin client api calls by @mayurinehate in https://github.com/datahub-project/datahub/pull/6863
• chore(ingest): loosen requirements file by @hsheth2 in https://github.com/datahub-project/datahub/pull/6867
• feat(ingest): upgrade pydantic version by @cccs-eric in https://github.com/datahub-project/datahub/pull/6858
• fix(elasticsearch): fixes out of order runId writes by @david-leifker in https://github.com/datahub-project/datahub/pull/6845
• chore(ingest): loosen additional requirements by @hsheth2 in https://github.com/datahub-project/datahub/pull/6868
• feat(ingest): bigquery/snowflake - Store last profile date in state by @treff7es in https://github.com/datahub-project/datahub/pull/6832
• docs(google-analytics): Correct grammatical error in README.md by @jx2lee in https://github.com/datahub-project/datahub/pull/6870
• feat(CI): add venv caching by @szalai1 in https://github.com/datahub-project/datahub/pull/6843
• feat(ingest/snowflake): handle failures gracefully and raise permission failures by @mayurinehate in https://github.com/datahub-project/datahub/pull/6748
• fix(runid): always update runid, except when queued by @david-leifker in https://github.com/datahub-project/datahub/pull/6876
• fix(ingest): conditionally include env in assertion guid by @hsheth2 in https://github.com/datahub-project/datahub/pull/6811
• chore(ci): update dependencies docs-website by @anshbansal in https://github.com/datahub-project/datahub/pull/6871
• feat(ui) - Add a custom error message for bulk edit to add clarity by @mkamalas in https://github.com/datahub-project/datahub/pull/6775
• docs(adding users): Refreshing the docs for adding new DataHub Users by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6879
• test(mce-consumer): mockbeans by @david-leifker in https://github.com/datahub-project/datahub/pull/6878
• feat(ingest): avoid embedding serialized json in metadata files by @hsheth2 in https://github.com/datahub-project/datahub/pull/6742
• refactor(gradle): move the local docker registry to common location by @david-leifker in https://github.com/datahub-project/datahub/pull/6881
• refactor(smoke): use env variables by @anshbansal in https://github.com/datahub-project/datahub/pull/6866
• fix(lint): pin pydantic version by @anshbansal in https://github.com/datahub-project/datahub/pull/6886
• refactor(docs): Correctly spell elasticsearch in docs by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6880
• fix(ingest): okta undefined variable error by @anshbansal in https://github.com/datahub-project/datahub/pull/6882
• fix(ci): reduce flakiness in add_users, siblings smoke test by @anshbansal in https://github.com/datahub-project/datahub/pull/6883
• fix(ingest): fall back to default table comment method for all Trino query errors by @marvin-roesch in https://github.com/datahub-project/datahub/pull/6873
• test(misc): misc test updates by @david-leifker in https://github.com/datahub-project/datahub/pull/6890
• deprecate(ingest): bigquery - Removing bigquery-legacy source by @treff7es in https://github.com/datahub-project/datahub/pull/6851
• chore(ingest): remove inferred args to MCPW, part 1 by @hsheth2 in https://github.com/datahub-project/datahub/pull/6819
• test(ingest/kafka-connect): make docker setup more reliable by @hsheth2 in https://github.com/datahub-project/datahub/pull/6902
• fix(ingest): profiling (bigquery) - Address biquery profiling query error due to timestamp vs data mismatch by @treff7es in https://github.com/datahub-project/datahub/pull/6874
• fix(cli): Make datahub quickstart work with latest docker compose in M1 by @pedro93 in https://github.com/datahub-project/datahub/pull/6891
• fix(cli): fix delete urn cli bug + stricter type annotations by @hsheth2 in https://github.com/datahub-project/datahub/pull/6903
• fix(ingest/airflow): reorder imports to avoid cyclical dependencies by @stijndehaes in https://github.com/datahub-project/datahub/pull/6719
• feat: remove jq requirement + tweak modeldocgen args by @hsheth2 in https://github.com/datahub-project/datahub/pull/6904
• chore(ingest): loosen pyspark and pydeequ deps by @hsheth2 in https://github.com/datahub-project/datahub/pull/6908
• docs(ingest/looker): fix typos + update lookml github action example by @hsheth2 in https://github.com/datahub-project/datahub/pull/6910
• fix(ingest/metabase): use card_id in dashboard to chart lineage by @ccpypy in https://github.com/datahub-project/datahub/pull/6583
• fix(es-setup): create data stream on non-aws by @szalai1 in https://github.com/datahub-project/datahub/pull/6926
• Adding missing Platform logos by @maggiehays in https://github.com/datahub-project/datahub/pull/6892
• feat(ingestion): PowerBI# Improve PowerBI source ingestion by @mohdsiddique in https://github.com/datahub-project/datahub/pull/6549
• Fix compose context for kafka-setup by @szalai1 in https://github.com/datahub-project/datahub/pull/6923
• feat(backend): Supporting Embeddable Previews for Dashboards, Charts, Datasets by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6875
• chore(deps): bump json5 from 2.2.1 to 2.2.3 in /docs-website by @dependabot in https://github.com/datahub-project/datahub/pull/6930
• chore(deps): bump json5 from 1.0.1 to 1.0.2 in /datahub-web-react by @dependabot in https://github.com/datahub-project/datahub/pull/6931
• fix(ci): managed ingestion test fix by @anshbansal in https://github.com/datahub-project/datahub/pull/6946
• feat(ingest): add include_table_location_lineage flag for SQL common by @hsheth2 in https://github.com/datahub-project/datahub/pull/6934
• feat(ingest): allow extracting snowflake tags by @frsann in https://github.com/datahub-project/datahub/pull/6500
• chore(ingest): unpin pydantic dep by @hsheth2 in https://github.com/datahub-project/datahub/pull/6909
• chore(ingest): partially revert pyspark dep from #6908 by @hsheth2 in https://github.com/datahub-project/datahub/pull/6954
• fix(ingest): use branch info when cloning git repos by @hsheth2 in https://github.com/datahub-project/datahub/pull/6937
• chore(ingest): remove inferred args to MCPW, part 2 by @hsheth2 in https://github.com/datahub-project/datahub/pull/6905
• fix(ingest/unity): simplify MCP generation and reporting by @hsheth2 in https://github.com/datahub-project/datahub/pull/6911
• chore(ci): parallelise build and test workflow to reduce time by @anshbansal in https://github.com/datahub-project/datahub/pull/6949
• fix(frontend): sasl.client.callback.handler.class by @szalai1 in https://github.com/datahub-project/datahub/pull/6962
• chore(react): remove outdated cypress tests and dependency by @anshbansal in https://github.com/datahub-project/datahub/pull/6948
• fix(ci): restrict GE to fix build issues by @anshbansal in https://github.com/datahub-project/datahub/pull/6967
• feat(queries): [Experimental] Allow customization of # of queries in Query tab via env var by @gabe-lyons in https://github.com/datahub-project/datahub/pull/6964
• feat(ingest/postgres): emit lineage for postgres views by @LucasRoesler in https://github.com/datahub-project/datahub/pull/6953
• feat(ingest/vertica): support projections and lineage in vertica by @vishalkSimplify in https://github.com/datahub-project/datahub/pull/6785
• fix(ingest): add missing dep for powerbi by @hsheth2 in https://github.com/datahub-project/datahub/pull/6969
• Docs fixes week of 12 22 by @laulpogan in https://github.com/datahub-project/datahub/pull/6963
• fix(ingest): unfreeze bigquery/snowflake column dataclass by @mayurinehate in https://github.com/datahub-project/datahub/pull/6921
• chore(frontend) Remove unused dependencies from package.json by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6974
• chore: misc fixes by @anshbansal in https://github.com/datahub-project/datahub/pull/6966
• feat(ingest/glue): emit s3 lineage for s3a and s3n schemes by @danielli-ziprecruiter in https://github.com/datahub-project/datahub/pull/6788
• fix(kafka-setup): Make kafka-setup run with multiple threads by @pedro93 in https://github.com/datahub-project/datahub/pull/6970
• feat(ingest): mark database_alias and env as deprecated by @hsheth2 in https://github.com/datahub-project/datahub/pull/6901
• fix(docs): Updating Tag, Glossary Term docs to point to correct GraphQL methods by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6965
• chore(deps): bump certifi from 2020.12.5 to 2022.12.7 in /metadata-ingestion/src/datahub/ingestion/source/feast_image by @dependabot in https://github.com/datahub-project/datahub/pull/6979
• fix(ingest): profiling - Fixing issue with the wrong timestamp stored in check by @treff7es in https://github.com/datahub-project/datahub/pull/6978
• config(quickstart): enable auto-reindex for quickstart by @david-leifker in https://github.com/datahub-project/datahub/pull/6983
• feat(privileges) - Create a privilege to manage glossary children recursively by @mkamalas in https://github.com/datahub-project/datahub/pull/6731
• chore(ingest): finish removing feast-legacy by @hsheth2 in https://github.com/datahub-project/datahub/pull/6985
• feat(ingest): add import descriptions of two or more nested messages by @wngus606 in https://github.com/datahub-project/datahub/pull/6959
• feat(docs) Add feature guide for Manual Lineage by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6933
• docs(rfc): Serialising GMS Updates with Preconditions by @mattmatravers in https://github.com/datahub-project/datahub/pull/5818
• fix(ingest/kafka-connect) support newer version of debezium by @jaegwonseo in https://github.com/datahub-project/datahub/pull/6943
• fix(docs): build and broken snowflake docs fix by @anshbansal in https://github.com/datahub-project/datahub/pull/6997
• fix(ingest): bigquery - views in case more than 1 datasets with views by @anshbansal in https://github.com/datahub-project/datahub/pull/6995
• fix(docs): Renaming Business Glossary Doc by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7001
• fix(ingest/snowflake): fix type annotations + refactor get_connect_args by @hsheth2 in https://github.com/datahub-project/datahub/pull/7004
• fix(docs): Changing the platform event topic name in kafka custom topic docs by @blankon123 in https://github.com/datahub-project/datahub/pull/7007
• fix(docs): fix name of privilege referenced in posts doc by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/7002
• fix(SSO): Correctly redirect to originally requested URL in SSO by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7011
• fix(ingest): remove dead code from tests by @hsheth2 in https://github.com/datahub-project/datahub/pull/7005
• feat(ingestion): Tableau # Embed links by @mohdsiddique in https://github.com/datahub-project/datahub/pull/6994
• feat(auth) Update auth cookies to have same-site none for chrome extension by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6976
• docs(website): DPG WIP by @maggiehays in https://github.com/datahub-project/datahub/pull/6998
• docs: resize datahub logo by @hsheth2 in https://github.com/datahub-project/datahub/pull/7014
• fix(kafka-setup): Remove reference to non-existing topic by @pedro93 in https://github.com/datahub-project/datahub/pull/7019
• fix(ingest): powerbi # use display name field as title for powerbi report page by @looppi in https://github.com/datahub-project/datahub/pull/7017
• feat(auth) Allow session ttl to be configurable by env variable by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/7022
• fix(ui): URL Encode all Entity Profile URLs by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7023
• fix(ui ingest): Fix test connection when stateful ingest is enabled by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7013
• docs(sso) move root user warning to earlier in SSO guides by @maggiehays in https://github.com/datahub-project/datahub/pull/7028
• fix(ingest/looker): add clarity in chart input parsing logs by @hsheth2 in https://github.com/datahub-project/datahub/pull/7003
• chore(ingest): remove duplicate data_platform.json file by @hsheth2 in https://github.com/datahub-project/datahub/pull/7026
• feat(ingestion): PowerBI # Remove corpUserInfo aspect ingestion by @mohdsiddique in https://github.com/datahub-project/datahub/pull/7034
• fix(metadata-models): remove unnecessary bin folder by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/7035
• fixing typos by @maggiehays in https://github.com/datahub-project/datahub/pull/7030
• feat(ingest): Ingest Previews for Looker Charts, Dashboards, and Explores by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6941
• fix(graphql):fix issue: autorender aspect could not be displayed on t… by @yangjiandan in https://github.com/datahub-project/datahub/pull/6993
• fix(config): adding quotes by @david-leifker in https://github.com/datahub-project/datahub/pull/7038
• fix(config): adding quotes by @david-leifker in https://github.com/datahub-project/datahub/pull/7040
• fix(ingest/bigquery): Turning some usage warning message to debug log as it caused confusion by @treff7es in https://github.com/datahub-project/datahub/pull/7024
• feat(ingest/vertica): Adding Vertica as source in Datahub UI by @Rajasekhar-Vuppala in https://github.com/datahub-project/datahub/pull/7010
• Removed a double set for two fields by @bda618 in https://github.com/datahub-project/datahub/pull/7037

New Contributors

• @marvin-roesch made their first contribution in https://github.com/datahub-project/datahub/pull/6873
• @stijndehaes made their first contribution in https://github.com/datahub-project/datahub/pull/6719
• @ccpypy made their first contribution in https://github.com/datahub-project/datahub/pull/6583
• @LucasRoesler made their first contribution in https://github.com/datahub-project/datahub/pull/6953
• @vishalkSimplify made their first contribution in https://github.com/datahub-project/datahub/pull/6785
• @wngus606 made their first contribution in https://github.com/datahub-project/datahub/pull/6959
• @jaegwonseo made their first contribution in https://github.com/datahub-project/datahub/pull/6943
• @blankon123 made their first contribution in https://github.com/datahub-project/datahub/pull/7007
• @yangjiandan made their first contribution in https://github.com/datahub-project/datahub/pull/6993
• @Rajasekhar-Vuppala made their first contribution in https://github.com/datahub-project/datahub/pull/7010

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.9.5...v0.9.6

DataHub v0.9.5

Released on 2022-12-23 by @jjoyce0510.

Release Highlights

Notice: This PR includes a fix for Single Sign-On (OIDC) that was introduced in the previous release, v0.9.4.

Important Release Notes

With this release, if you are using Neo4J as your graph implementation, you need to set: GRAPH_SERVICE_DIFF_MODE_ENABLED=false

For GMS (or MAE Consumer for standalone mode).

User Experience

• Manual Lineage is LIVE! You can now add and remove lineage between entities in the Lineage Visualization screen, making it easier than ever to manage the complex relationships between your data resources.

• Our new Views feature makes it easy to create curated sets of Entities within DataHub. This is a great way to start to isolate the entities that matter most, and provide your DataHub end-users with a streamlined view of the assets that are relevant to their use cases. See the original demo video.

• In-App Product Tours are here! When logging into DataHub and/or visiting a new page type for the first time, new users will be prompted with a helpful walkthrough of core functionality to get them familiar with the platform. We’ll continue to add modules as we roll out new features!

• Automatically send updates to Slack and/or Microsoft Teams when changes are made within DataHub by leveraging our the new Slack and Teams Actions.

Metadata Ingestion

We’re continuing to improve the user experience for UI-based ingestion for the following sources:

• DataBricks Unity Catalog
• dbt Cloud
• MySQL
• Trino/Presto
• Microsoft SQL Server
• MariaDB

If you’re just getting started with UI-based Ingestion, check out our new BigQuery & Snowflake guides.

Stateful ingestion is now supported for Iceberg (thanks for the contrib, @cccs-Dustin!) and LDAP (thanks for the contrib, @bda618!)

What's Changed

• feat(ingest): add failure/warning counts to ingest_stats by @hsheth2 in https://github.com/datahub-project/datahub/pull/6823
• refactor(ingest): clean up pipeline init error handling by @hsheth2 in https://github.com/datahub-project/datahub/pull/6817
• fix(ingest): exclude ztsd from uber jar to prevent jni conflicts with spark by @danielli-ziprecruiter in https://github.com/datahub-project/datahub/pull/6787
• feat(ingest/bigquery): add option to enable/disable legacy sharded table support by @treff7es in https://github.com/datahub-project/datahub/pull/6822
• fix(ingest): support patches in auto_status_aspect by @hsheth2 in https://github.com/datahub-project/datahub/pull/6827
• fix(ci): reduce flakiness views select test by @anshbansal in https://github.com/datahub-project/datahub/pull/6821
• refactor(ingest): clean up exception types by @hsheth2 in https://github.com/datahub-project/datahub/pull/6818
• fix(ingest): fixed snowflake oauth ingestion not using role attribute… by @daguito81 in https://github.com/datahub-project/datahub/pull/6825
• refactor(ingestion): Browse Paths Upgrade V2 Feast & Sagemaker by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6002
• fix(lineage) Fix lineage viz with multiple siblings by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6826
• fix(pac4j-oidc): add verifier parameter by @david-leifker in https://github.com/datahub-project/datahub/pull/6835
• feat(ingest): extract kafka topic config properties as customProperties by @mayurinehate in https://github.com/datahub-project/datahub/pull/6783
• docs: Incorrect import statement fixed in example by @mirac-cisco in https://github.com/datahub-project/datahub/pull/6838
• feat(ingestion): support lineage for delta lake writes by @danielli-ziprecruiter in https://github.com/datahub-project/datahub/pull/6834
• feat(ui): Support defining the ID for Glossary Term and Glossary Term Group in UI by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6830
• feat(ci): add cypress test ui based ingestion by @anshbansal in https://github.com/datahub-project/datahub/pull/6769
• feat(ui): sortable domain list by @looppi in https://github.com/datahub-project/datahub/pull/6736
• fix(ci): add labels based on more folders by @anshbansal in https://github.com/datahub-project/datahub/pull/6840
• fix(ingest): kafka ingest task hand up with error bootstrap server by @wangsaisai in https://github.com/datahub-project/datahub/pull/6820
• fix(ingest): Fixing Kafka source linting by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6844
• fix(ingestion) Inject pipeline_name into recipes at runtime by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6833
• feat(ingest): add db/schema properties hook to SQL common by @hsheth2 in https://github.com/datahub-project/datahub/pull/6847
• fix(oidc): fix oidc authentication loop by @david-leifker in https://github.com/datahub-project/datahub/pull/6848
• docs(confluent): add details for actions pod for confluent by @RyanHolstien in https://github.com/datahub-project/datahub/pull/6810
• feat(ingestion): Business Glossary# Add domain support in GlossaryTerm ingestion by @mohdsiddique in https://github.com/datahub-project/datahub/pull/6829
• fix(ingest/looker): handle missing label fields by @hsheth2 in https://github.com/datahub-project/datahub/pull/6849
• refactor(ui): Misc domains improvements by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6850

New Contributors

• @daguito81 made their first contribution in https://github.com/datahub-project/datahub/pull/6825
• @mirac-cisco made their first contribution in https://github.com/datahub-project/datahub/pull/6838

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.9.4...v0.9.5

[Known Issues] DataHub v0.9.4

Released on 2022-12-20 by @maggiehays.

View the release notes for [Known Issues] DataHub v0.9.4 on GitHub.

DataHub v0.9.3

Released on 2022-11-30 by @maggiehays.

View the release notes for DataHub v0.9.3 on GitHub.

DataHub v0.9.2

Released on 2022-11-04 by @maggiehays.

View the release notes for DataHub v0.9.2 on GitHub.

DataHub v0.9.1

Released on 2022-10-31 by @maggiehays.

View the release notes for DataHub v0.9.1 on GitHub.

DataHub v0.9.0

Released on 2022-10-11 by @szalai1.

View the release notes for DataHub v0.9.0 on GitHub.

DataHub v0.8.45

Released on 2022-09-23 by @gabe-lyons.

View the release notes for DataHub v0.8.45 on GitHub.

DataHub v0.8.44

Released on 2022-09-01 by @jjoyce0510.

View the release notes for DataHub v0.8.44 on GitHub.

DataHub v0.8.43

Released on 2022-08-09 by @maggiehays.

View the release notes for DataHub v0.8.43 on GitHub.

v0.8.42

Released on 2022-08-03 by @gabe-lyons.

View the release notes for v0.8.42 on GitHub.

v0.8.41

Released on 2022-07-15 by @anshbansal.

View the release notes for v0.8.41 on GitHub.

v0.8.40

Released on 2022-06-30 by @gabe-lyons.

View the release notes for v0.8.40 on GitHub.

v0.8.39

Released on 2022-06-24 by @maggiehays.

View the release notes for v0.8.39 on GitHub.

[!] DataHub v0.8.38

Released on 2022-06-09 by @jjoyce0510.

View the release notes for [!] DataHub v0.8.38 on GitHub.

[!] DataHub v0.8.37

Released on 2022-06-09 by @jjoyce0510.

View the release notes for [!] DataHub v0.8.37 on GitHub.

DataHub V0.8.36

Released on 2022-06-02 by @treff7es.

View the release notes for DataHub V0.8.36 on GitHub.

[!] DataHub v0.8.35

Released on 2022-05-18 by @dexter-mh-lee.

View the release notes for [!] DataHub v0.8.35 on GitHub.

v0.8.34

Released on 2022-05-04 by @maggiehays.

View the release notes for v0.8.34 on GitHub.

DataHub v0.8.33

Released on 2022-04-15 by @dexter-mh-lee.

View the release notes for DataHub v0.8.33 on GitHub.

DataHub v0.8.32

Released on 2022-04-04 by @dexter-mh-lee.

View the release notes for DataHub v0.8.32 on GitHub.

DataHub v0.8.31

Released on 2022-03-17 by @dexter-mh-lee.

View the release notes for DataHub v0.8.31 on GitHub.

Datahub v0.8.30

Released on 2022-03-17 by @rslanka.

View the release notes for Datahub v0.8.30 on GitHub.

DataHub v0.8.29

Released on 2022-03-10 by @shirshanka.

View the release notes for DataHub v0.8.29 on GitHub.

DataHub v0.8.28

Released on 2022-03-07 by @shirshanka.

View the release notes for DataHub v0.8.28 on GitHub.

DataHub Release Candidate v0.8.28 (rc1)

Released on 2022-03-05 by @shirshanka.

View the release notes for DataHub Release Candidate v0.8.28 (rc1) on GitHub.

Release Candidate v0.8.28

Released on 2022-03-05 by @shirshanka.

View the release notes for Release Candidate v0.8.28 on GitHub.

DataHub v0.8.27

Released on 2022-02-23 by @shirshanka.

View the release notes for DataHub v0.8.27 on GitHub.