A year and a half ago, Microsoft released Fabric to public preview during the Build conference. For many, including myself, it seemed to come out of nowhere. Fabric was Microsoft’s entry into the race to harness enterprise data, fueling the start of the AI era.

Six months later, at Ignite 2023, Fabric became generally available. At Zure, we’ve had the opportunity to witness, alongside our customers, the rapid development of Fabric. According to Satya Nadella, it is “perhaps the biggest launch of a data product from Microsoft since SQL Server”.

Now almost a year has passed and Ignite 2024 is here to reveal new features and to showcase new features and improvements to Microsoft Fabric. Over the last year, Microsoft has maintained an aggressive development pace, releasing around 150 features – that’s about 14 features per month. This excludes smaller features and improvements deployed simultaneously. However, the product isn’t perfect; the known issues list is still substantial. While our team at Zure encounters unexpected problems time to time, it’s impressive how much Fabric has improved from its early days. I trust that the foundation is solid and will continue to evolve. Only time will tell where we will be a year from now.

So, what new insights did we gain from this year’s Microsoft Ignite? Let’s dive in! 🚀

Fabric as an AI-fueled data platform

AI remains the top buzzword, and it’s integral to how Microsoft positions Fabric as an AI-fueled data platform solution. By rebranding the principles of Fabric, the main message is that Microsoft Fabric enables data experts to achieve more in less time and lowers the bar for data citizens to become more data-driven in their everyday work with the help of AI and Copilot. Here are the new focus points for Fabric:

🌟 AI-powered data platform
🌟 Open and AI-ready data lake
🌟 AI-enabled business users
🌟 Mission-critical foundation

Analytical AND Transactional Workloads in the Same Place?

Microsoft introduced the Fabric Database class, initially bringing the SQL database into the Fabric platform. This new class allows applications to be closer to analytical solutions when databases are hosted in the same environment, reducing delays with automatic data replication to OneLake. In OneLake, data is replicated in delta format, providing the same SQL endpoint as the other items. This action is almost real-time and isolates these two workloads from each other, bringing analytical and transactional closer to each other than before. It is extremely interesting and can have a dramatic impact on future architectural patterns. This will need a closer look and in-depth blog post soon! 🔎

Real-Time Intelligence reaches general availability

Introduced in May 2024, Real-Time Intelligence allows users to ingest and process streaming data while delivering immediate insights. At Ignite, it was announced that RTI is now generally available, and new features were previewed:

• OneLake and job events in Real-Time Hub: Enables automated and efficient reactions to data changes.
• CI/CD capabilities: Features like REST APIs, Git integration, and deployment pipelines simplify development workflows.
• OneLake Query Acceleration: I mentioned this in the previous post and wondered when it will become available. Looks like now is the time! The core idea is to make lakehouse-based tables compatible with the KQL database and join together analytical and streaming data quickly.

These additions streamline the creation of real-time data solutions and improve responsiveness across Fabric components.

Simplified data management with OneLake Catalog

Managing numerous workspaces and items within the Fabric portal, especially in larger environments, can be cumbersome. To simplify this, Microsoft introduced the OneLake catalog, an evolution of the OneLake data hub. It addresses challenges in managing large-scale environments by offering two views:

• Explore View: Provides a unified overview of all accessible workspaces and items, reducing the need for navigation.
• Govern View: Designed for data stewards, this feature (coming soon) enables better management of endorsements, labels, and asset types across distributed environments.

This allows for the management of data assets distributed across different workspaces from a single view. The Explore view is available now, and the Govern view will be coming soon.

Open Mirroring expands data integration options

Microsoft announced that Azure SQL Database is now generally available and SQL Managed Instance is in the preview phase as a source for mirroring. Additionally, the Open Mirroring concept has been introduced, removing the restriction to use specific technology for implementing mirroring in the Fabric platform. Open Mirroring simplifies data syncing between applications and Fabric, allowing a wide range of solutions to be integrated into a single platform.

AI Hype – advancements in AI features

On the AI front, Microsoft had previously released the AI Skills capability, which offers users a personal chatbot experience for specific datasets located in Fabric. Now, Microsoft has announced that in addition to lakehouse and warehouse items, AI Skills can also utilize KQL databases or semantic models as a base for data search. The support for semantic models is particularly useful, expanding the usability of AI Skills as many customers already have these models in place.

Lastly, AI functions were mentioned and are coming soon. This gives a simplified way to implement the common AI enrichments like summarization or sentiment analyzing. This opens up the discoverability part for data engineer to understand better the underlying and push new insights for the data consumers.

Staying up-to-date with Fabric

These updates, alongside many others, are detailed in the official Book of News. Microsoft shows no signs of slowing down its development pace for Fabric, ensuring it remains a leading platform in the AI-driven data landscape.