LinkedIn DataHub Project Updates (February 2021 Edition)
Background
LinkedIn DataHub is an open-source metadata platform that I founded and architected at LinkedIn and is in wide use at many companies. We hold monthly town-hall meetings to share new developments, hear from the community about their adoption journeys, and plan together for the future. I’m starting a journal to “catalog” the evolution of the project for posterity in a way that a video recording or a git commit log cannot. This post captures the developments for the month of February 2021.
Community Update
More than 50 PR-s were submitted and merged over the last 30 days! We had almost 50 people attending the monthly town hall. Our Slack community is growing fast. Join us to talk metadata and subscribe to our YouTube channel.
The big contributions in the last month were:
- Formalized support for GraphQL in DataHub. It supports embedding in the Play frontend server as well as a standalone service. John Joyce from Acryl and Arun Vasudevan from Expedia have been leading the charge on this.
- New React application is now ready for beta-testing
- New Python-based ingestion framework
Read on to find out more!
New Search and Discovery App in React
Extensible UI, delightful experience
The original DataHub UI was built on the Ember framework, a Javascript framework that is used at many companies including LinkedIn. However, Ember is not as popular as other frameworks like React among the developer community; and this impacted the ability of new contributors to ramp up and contribute. On top of that, for the frontend code, LinkedIn was following a “develop internally and then release into open-source” model; making it hard for the project to accept external contributions easily.
To change the status quo, we decided to incubate a new React application in the project. The application has been in incubation for a month and was demo-ed at the town-hall event by John Joyce. It features a fresh look, a re-designed search and browsing experience, and is targeted to support a much higher level of extensibility. The decision is already reaping dividends as we got our first UI contribution from the community to implement logout functionality.
Or click here to finish reading….
Connect with DataHub
Join us on Slack • Sign up for our Newsletter • Follow us on Twitter
