semantic-link 0.13.0

Semantic link for Microsoft Fabric

Semantic link is a feature that allows you to establish a connection between Power BI datasets and Synapse Data Science in Microsoft Fabric.

The primary goals of semantic link are to facilitate data connectivity, enable the propagation of semantic information, and seamlessly integrate with established tools used by data scientists, such as notebooks.

Semantic link helps you to preserve domain knowledge about data semantics in a standardized way that can speed up data analysis and reduce errors.

Package (PyPi) | API reference documentation | Product documentation | Samples

By downloading, installing, using or accessing this distribution package for semantic link, you agree to the Terms of Service.

This package has been tested with Microsoft Fabric.

Getting started

Prerequisites

• A Microsoft Fabric subscription. Or sign up for a free Microsoft Fabric (Preview) trial.
• Sign in to Microsoft Fabric.
• Create a new notebook or a new spark job to use this package. Note that semantic link is supported only within Microsoft Fabric.

About the semantic link packages

The functionalities for semantic link are split into multiple packages to allow for a modular installation. If you want to install only a subset of the semantic link functionalities, you can install the individual packages instead of the semantic-link meta-package. This can help solve dependency issues. The following are some of the available packages:

• semantic-link - The meta-package that depends on all the individual semantic link packages and serves as a convenient way to install all the semantic link packages at once.
• semantic-link-sempy - The package that contains the core semantic link functionality.
• semantic-link-functions-holidays - A package that contains semantic functions for holidays and dependence on holidays.
• semantic-link-functions-geopandas - A package that contains semantic functions for geospatial data and dependence on geopandas.
• ...

Install the semantic-link meta package

For Spark 3.4 and above, Semantic link is available in the default runtime when using Fabric, and there's no need to install it. If you're using Spark 3.3 or below, or if you want to update to the most recent version of Semantic Link, you have two options:

• To install the most recent version semantic-link in your notebook kernel by executing this code in a notebook cell:

  %pip install -U semantic-link
  

• Alternatively, you can add semantic link to your Fabric environments directly. For more information, see library management in Fabric environments.

Key concepts

SemPy offers the following capabilitites:

• Connectivity to Power BI
• Connectivity through Power BI Spark native connector
• Data augmentation with Power BI measures
• Semantic propagation for pandas users
• Built-in and custom semantic functions

Change logs

0.13.0

• New modules:
  • sempy.fabric.lakehouse
    • New functions:
      • delete_lakehouse: Delete a lakehouse in the specified workspace.
      • resolve_lakehouse_name: Resolve the name of a lakehouse in the specified workspace.
      • resolve_lakehouse_id: Resolve the ID of a lakehouse in the specified workspace.
      • list_lakehouse_tables: List all tables in a lakehouse and their properties.
      • vacuum_lakehouse_tables: Run the VACUUM function over the specified lakehouse tables.
    • Mirrored function (re-exported from sempy.fabric): create_lakehouse.
  • sempy.fabric.report
    • New functions:
      • clone_report: Clone a report in the specified workspace.
      • rebind_report: Rebind one or more reports to a semantic model.
    • Mirrored function (re-exported from sempy.fabric): list_reports.
  • sempy.fabric.semantic_model
    • New functions:
      • refresh_semantic_model: Refresh a semantic model synchronously with options to visualize the refresh progress and collect SSAS traces.
      • cancel_refresh: Cancel the specific refresh of a semantic model.
      • list_refresh_requests: Poll the status of refresh requests for a given dataset using Enhanced refresh with the Power BI REST API.
    • Mirrored functions (re-exported from sempy.fabric): refresh_dataset, list_refresh_requests, get_refresh_execution_details, list_datasets, resolve_dataset_id, resolve_dataset_name, resolve_dataset_name_and_id.
  • sempy.fabric.sql_endpoint
    • New functions:
      • list_sql_endpoints: List all SQL endpoints in the specified workspace.
      • get_sql_endpoint_id: Get the SQL endpoint ID of the specified warehouse.
      • get_sql_endpoint_connection_string: Get the SQL endpoint connection string of the specified warehouse.
      • refresh_sql_endpoint_metadata: Refresh the metadata of a SQL endpoint.
  • sempy.fabric.spark
    • New functions:
      • get_spark_settings: Get the Spark settings for the specified workspace.
      • update_spark_settings: Update the Spark settings for the specified workspace.
      • get_fabric_runtime: Get the default Fabric runtime version for the specified workspace.
      • set_fabric_runtime: Set the default Fabric runtime version for the specified workspace.
• New functions under sempy.fabric:
  • get_sku_size: Get the SKU size of the specified workspace.
  • get_item_definition: Get the definition of an item in the specified workspace.
• Updated functions under sempy.fabric:
  • resolve_item_id:
    • Renamed parameter item_name to item to accept both item name or item ID as the first parameter.
    • Renamed optional parameter type to item_type.
    • Using named parameter item_name or type will show deprecation warning from now on.
  • resolve_item_name:
    • Renamed parameter item_id to item to accept both item name or item ID as the first parameter.
    • Renamed optional parameter type to item_type.
    • Using named parameter item_id or type will show deprecation warning from now on.
  • list_items:
    • Renamed optional parameter type to item_type.
    • Using named parameter type will show deprecation warning from now on.
• Bug fixes:
  • Fix set_service_principal for SNI enabled application.
  • Fix correctly populating null measures for evaluate_measure.

0.12.3

• add TOMWrapper.all_functions: support retrieving all functions in a semantic model
• update dependency versions

0.12.2

• fix dependencies

0.12.1

• update .NET Microsoft.AnalysisServices lib to 19.104.1
• update .NET Microsoft.AnalysisServices.AdomdClient lib to 19.104.1

0.12.0

• new: support authentication with Azure Identity client library
• new: support authentication with service principals

0.11.2

• new: support read-write connection for evaluate_measure, evaluate_dax and execute_xmla via use_readwrite_connection parameter
• new: support strict typed schema for TraceConnection.disover_event_schema via strict_types parameter
• fix: fix columns population error like "ApplicationContext" in customized event schemas for TraceConnection

0.11.1

• new: support create_lakehouse with enable_schema parameter
• fix: Fix populating source expression to "Query" column for PolicyRange partitions in list_partitions

0.11.0

• new: support Fabric folder (list, resolve, create, rename, move, delete)
• new: support ISO language codes for translate_semantic_model
• added: translation option in run_model_bpa via language parameter
• ​added​​: authentication control in evaluate_dax via role and effective_user_name parameters
• fixed: date fromatting rule for BPA

0.10.2

• fix setup.py: fix pandas dependency for Fabric Spark 3.3
• fix _utils/_log.py: improve robustness for mds attributes reading.

0.10.1

• fix _utils/_log.py: correct dict key for mds attributes

0.10.0

• add translate_semantic_model: support translating semantic models
• update run_model_bpa: support analyzing DAX expression on calculated tables, calculated columns, and calculation items
• update list_table: add optional parameter include_internal to include internal tables
• update model_memory_analyzer: support analyzing size of internal tables
• fix model_memory_analyzer: escaping dax tables and columns in list_relationships and model_memory_analyzer

0.9.3

• fix: activate Model Summary tab in model_memory_analyzer by default

0.9.2

• fix: run run_model_bpa/model_memory_analyzer on empty dataset
• fix: model_memory_analyzer list missing relationships

0.9.1

• add retry_config parameter to BaseRestClient
• add dataset parameter to create_tom_server
• fix: escaping "/" in workspace url
• fix: get fabric context error
• fix: trace add event schema for ActivityId and RequestId
• fix: missing rows calculation logic on direct lake mode models
• fix: occasional dataset not found error in list_tables
• fix: bpa rules for unescaped object names

0.9.0

• add model_memory_analyzer: support displaying VertiPaq statistics about the semantic model
• add run_model_bpa function: support displaying Best Practice Analyzer statistics about the semantic model
• update connect_semantic_model: support using either name or ID for workspace and dataset
• fix evaluate_dax for object columns: cast object columns to strings
• fix TOMWrapper all_unqualified_column_dependencies: correct the input type to be Measure

0.8.5

• Fix refresh_tom_cache: addressed the issue when refreshing the workspace with different identifiers (e.g., workspace name and workspace ID)
• Fix TOMWrapper: addressed the lineage compactibility error during TOMWrapper initialization
• Upgrade Microsoft.AnalysisServices.AdomdClient NuGet dependency to 19.87.7

0.8.4

• add TOMWrapper for semantic model
• delay sklearn import to import startup latency

0.8.3

• fix: race conditions in sempy initialization during multithreading

0.8.2

• Telemetry Updates, bugfixes

0.8.1

• fix inaccuracy in the row count for certain scenarios when using the list_tables function with extended=True.

0.8.0

• add list_dataflow_storage_accounts
• fix list_dataflows: fix listing wrong results
• fix overflowing column metadata resolving warnings in dataset clients
• update fabric.read_table: support setting the import option by onelake_import_method parameter
• update FabricDataFrame.to_lakehouse_table: support setting the export option by method parameter

0.7.7

• fix list_partitions: records per segment calculation
• Added resolve_dataset_id and resolve_dataset_name

0.7.6

• update evaluate_dax: allow limiting number rows
• fix create_notebook: supported resolving default lakehouse from another workspace
• fix api doc: removed broken xrefs
• fix get_artifact_id for high concurrency

0.7.5

• FabricRestClient & PowerBIRestClient: support waiting for long-run operations
• FabricRestClient & PowerBIRestClient: support paged responses
• Added resolve_item_id and resolve_item_name
• evaluate_dax: support reading data from semantic models with read-only access

0.7.4

• internal bug fixes

0.7.3

• add delta_column_mapping_mode parameter to FabricDataFrame.to_onelake_table
• update find_relationships: swap from/to for relationships to align with PowerBI
• make sure users can execute DAX against semantic models they have access to AND not have access to the workspace
• support jupyter runtime
• update list_columns : added missing workspace parameter
• fix list_partitions: record / segment computation
• fix list_tables_duplicates
• fix list_tables(extended=True)
• fix broken doc links

0.7.2

• list_* (additional_xmla_properties): handle property names that might fail for some rows
• fix list_tables

0.7.1

• fix list_annotations

0.7.0

• add create_tom_server
• add additional_xmla_properties argument to all applicable list_* functions
• add list_annotations
• update list_columns: alternate columns/tables
• update list_relationships: add extended argument
• update list_hierarchies: add extended argument
• update list_partitions: add extended argument
• update list_measures: add additional columns
• fix plot_relationship_metadata: arrows for relationships point in the same direction as PowerBI
• fix list_datasources

0.6.0

• add list_datasources
• add list_dataflows
• add list_apps
• add list_list_gateways
• add list_tables
• add list_calcuation_items
• add list_columns
• add list_perspectives
• introduce the "extended" flag to query DMVs with more information (e.g. table size)
• add additional xmla properties
• update capacity id to lower case
• make FabricDataFrame creation more robust
• fix list_translations

Next steps

View our Samples