Our inside story
On our blog we'll start with a definition of why we created Schemabook and then go deeper into the features we develop over time.
When it comes to data within an organization, there are typically two main groups of stakeholders: the producers and the consumers. In many organizations, there is a disconnect between these two groups, with data often being thrown over the fence from one to the other without any collaboration.
In a previous organization, this lack of collaboration was evident in the search term analytics department. The website had a typical search form. Customers would search for products they were interested in. The analytics department was responsible for distilling this data down into a consumable report for the executive team. The report included break down by geographic region. The idea was to identify what was trending in each major region.
In a typical ETL setup, this report was stale by a day at best. Essentially, the team that implemented the search functionality would export data for a 24 hour period every night at midnight. This didn’t allow the flexibility to respond the executives were looking for. If a sudden spike occurred for a product in a specific region, the team wanted to respond within the hour, not the next day. So, the team responsible for the report needed to ingest the data on a faster cadence, perhaps in real time. However, the team didn’t know who implemented search. All they knew was that data was landing in a table in the lake every 24 hours. They didn’t know the producers. They didn’t know how to request the data on a faster cadence.
This common scenario highlights the need for a platform like Schemabook, which allows data producers to declare their data and set expectations for delivery, such as frequency and storage. Consumers can then easily search for and discover the data they need, defining themselves as stakeholders and staying informed of any changes. Schemabook facilitates collaboration between producers and consumers, ensuring that data is actionable and costs are minimized.
By focusing on stakeholders rather than specific technologies, Schemabook aims to improve data collaboration within organizations, ultimately leading to more effective and efficient data usage.