Microsoft Fabric: Automatically Synchronize Data from Semantic Models into OneLake

This feature is incredibly exciting, and it feels like it hasn’t received the attention it deserves amidst all the other recent updates. There’s certainly a lot of innovative updates in the Fabric platform coming every day!

 

In this post, I’ll guide you on how to automatically export data from your semantic models to Delta tables in OneLake. You only need to set the configuration up once, and Fabric will take care of exporting and updating the data for you—no need for pipelines, dataflows, or scripts. It’s that simple!

 

Prerequisites

 

OneLake integration for semantic models is exclusively available with Power BI Premium P and Microsoft Fabric F subscription tiers. This feature isn’t accessible through Power BI Pro, Premium Per User licenses, or Power BI Embedded A/EM plans.

 

1. Workspace with at least one semantic model in import mode hosted on a Power BI Premium or Fabric capacity.

 

2. Option Large semantic model storage format enabled for the particular model.

 

Permissions

 

The user should have a contributor role in order to be able to create shortcuts linking the semantic model delta tables into Lakehouse.

 

 

1. Semantic model settings

 

Go to the settings pane of the semantic model which date you would like to export to OneLake and make sure both options: Large semantic model storage format  and OneLake Integration are enabled.

 

 

In addition, make sure that the following settings are enabled in the Power BI Admin portal:

 

1. Semantic models can export data to OneLake: If this option is turned off, users cannot enable OneLake integration for their semantic models, and any models already set up for OneLake integration will stop exporting import tables to OneLake. By default, OneLake integration is enabled for the entire organization.

 

2. Users can store semantic model tables in OneLake: Allowing specific or all users to configure OneLake integration for their models. If this feature is disabled for a particular user, they will no longer be able to enable OneLake integration. However, any semantic models they’ve already set up for OneLake integration will continue exporting import tables.

 

Both options are enabled by default and can be managed by global and tenant administrators.

 

2. Refresh

 

After the OneLake integration has been enabled, a manual or scheduled refresh is needed to be performed on the semantic model in order for the data to be written to a Delta table in OneLake.

 

3. Import data into Lakehouse

 

Next step is to link the exported data into your lakehouse, by creating shortcuts.

 

1. Go to your Lakehouse.

 

2. Under Tables, select the three dots and select New shortcut

 

 

3. In the pop-up window, select Microsoft OneLake as a source.

 

4. If the refresh of your semantic model was successful and you have proper permissions on it, you should be able to see the semantic model in the list of available data sources to choose from

5. After selecting your semantic model, you can select which tables from it you would like to create a shortcut in your Lakehouse. You can select a specific table or create them all.

 

6. Once the creation process is complete, you can see your newly created shortcuts under the Tables section. Now you can do all the necessary operations over them, as they are regular tables in Lakehouse (i.e. querying through PySpark notebook).

 

 

7. Upon each model refresh, the latest data will be exported to the OneLake, thus the user will always have the newest data in the Lakehouse.

 

Limitations

 

There are certain limitations that should be considered in order to understand whether this functionality fulfills your business requirement and whether you should proceed with its implementation. The most important ones are:

 

• Measures, DirectQuery tables, hybrid tables, calculation group tables, and system managed aggregation tables can’t be exported to Delta format tables.

 

• Only a single version of the delta tables is exported and stored on OneLake. Old versions of the delta tables are deleted after a successful export. Meaning that there is no mechanism for storing historical data or versioning. To do this, a workaround can be implemented.

 

Keeping Your Data Secure in the Cloud

How is information supposed to be stored? Is it better to store information locally or on the cloud? How secure is your information anyway? These questions do not often have an easy answer.

 

We used to use disks, drives, and all sorts of equipment for storing our data, but today the cloud has become a place where both people and companies keep and exchange information on.

 

Lately we have been getting used to using the cloud more and more in order to permanently store all sorts of information, even the most sensitive ones. We trust the cloud more and more so the question arises.

 

How Sure Can We Be that Our Information Is Safe and Secure Out there on the Cloud?

 

This is both a legitimate and an illegitimate concern. It is legitimate because most of us feel safer when we hold something away from harm ourselves. To truly be out of harm, information needs to be kept in a closed system, with no outside influence. That is sometimes OK with pictures on a USB drive, but it certainly is not OK with information critical for many departments in a company for instance.

 

In the modern age of doing business, information needs to be available instantaneously, and nothing is faster than keeping them on the cloud. You need to think of information as money (which in many cases it is), and think of the cloud as a bank that holds the most important information. Most of us keep our money in the bank considering it secure, since we know banks are often safer than our homes when it comes to it. It is quite similar with the cloud. This is why, same as with the bank you choose, it is really important to choose who stores and handles your most critical information.

 

This is where Microsoft Dynamics AX comes handy. While you can read about the benefits of an ERP software for most businesses in our previous post, here you can find out why Microsoft Dynamics AX is the most secure ERP software out there.

 

Dynamics AX is built using the Security Development Lifecycle, which is a mandatory process that embeds security requirements into every phase of development. Identifying users and groups of users is made easy and that way you can assign and revoke access authorization easily.

 

Security-hardened infrastructure is used to connect data centers which helps block unauthorized traffic to and within those data centers, using a variety of technologies. Microsoft Anti malware helps protect the information against online threats. Intrusion detection, distributed denial of service (DDoS) attack prevention, regular penetration testing, and data analytics are some of the methods used to help mitigate threats.

 

In terms of privacy, you own and control your data. It can be accessed at any time and for any reason without the help or approval of Microsoft. The data will never be used for any purpose by Microsoft whatsoever. Steps are taken to prevent unauthorized access. Furthermore, Dynamics AX provides your own database to maximize the security and integrity of your data.

 

Government or law enforcement must follow applicable legal processes in order to gain access to your data even for national security purposes. In case that happens Microsoft always attempts to redirect the request directly to our customer. Unless legally prohibited from doing so, Microsoft will promptly notify you of any such request.

 

Microsoft is committed to achieving and maintaining compliance with leading data protection and privacy laws applicable to cloud services. As an ERP system, Dynamics AX includes features that will help organizations meet specific tax, accounting, or financial reporting requirements.

 

Finally, your data is stored transparently, and you always know where it is stored. Moreover, it is disclosed how the data storage location is determined.

 

These are amongst the reasons why choosing Microsoft Dynamics AX is the safest cloud based ERP software out there. It is because a successful attack against an ERP system can be devastating for any company. Especially for businesses with highly sensitive information, a company always need to pick the safest option.

 

Contact us to find the best and safest AX solution for your business.