Sending API Data to Fabric Real-Time Intelligence with Notebooks
- Vahid
- Feb 16
- 8 min read
Microsoft Fabric is changing the way we handle data by providing a unified analytics platform. One of its most powerful capabilities is Real-Time Intelligence, which allows us to capture and analyze live data streams instantly. Whether you’re working with stock prices, IoT sensors, or any other live data, streaming data into Fabric opens up a world of possibilities for real-time analytics and automation.
In this blog post, we’ll explore how to fetch data from an API using a Fabric Notebook and send it to EventStream, enabling seamless real-time processing in Fabric. We’ll walk through the entire process—from making API calls to structuring the data and streaming it into Fabric.
By the end of this guide, you’ll have a working setup that pulls live data from an API every few seconds and sends it to Real-Time Intelligence in Fabric for further analysis.
Why Use Notebooks for Real-Time Data in Fabric?
Microsoft Fabric provides a unified analytics platform, and Notebooks play a crucial role in automating data ingestion, transformation, and streaming. Unlike traditional ETL tools, Fabric Notebooks allow you to write and execute Python or Spark code directly within Fabric, making it an excellent choice for real-time data processing.
By using Fabric Notebooks, you can:
Fetch live data from APIs and process it in real time.
Add value to the data like timestamps, formatting, or filtering.
Send the transformed data directly to EventStream for real-time analytics.
Automate the entire process by running the Notebook at regular intervals.
Why Stream API Data into Fabric?
APIs provide a gateway to real-time information. Whether it's crypto prices, stock market data, IoT sensor readings, or weather updates, APIs give you access to continuously updating datasets. However, simply fetching the data is not enough—we need to process and stream it into Fabric to unlock its full potential.
By sending API data to Fabric Real-Time Intelligence, we can:
Monitor data streams live for instant decision-making.
Run real-time analytics using KQL (Kusto Query Language).
Create real-time dashboards
Send the data to Power BI for real-time reports.
Why Use EventStream for Real-Time Data?
Fabric’s EventStream acts as a real-time data ingestion pipeline. Instead of storing static data in a database, EventStream allows you to continuously receive, process, and route streaming data to different Fabric components like KQL databases or lakehouese.
How This Works in Our Use Case
In this blog post, we will fetch live crypto price data from Binance API using a Fabric Notebook, then enrich it with timestamps before sending it to EventStream. Finally, we will ensure the notebook automates this process every 2 seconds, creating a real-time data pipeline inside Microsoft Fabric.
Prerequisites: What You Need Before Starting
Before implementing the solution, it is important to ensure that all the necessary components are in place. This section outlines the key requirements for successfully setting up a Fabric Notebook, connecting to an API, and streaming real-time data into Microsoft Fabric’s EventStream.
Microsoft Fabric Access
To work with Notebooks and EventStream, you need access to a Microsoft Fabric workspace that lets you use Real-Time intelligence. If you are unsure whether you have the required access, you can verify it by logging into the Microsoft Fabric/ Power BI Service and checking your workspace settings. If Fabric is not available in your environment, you may need to request access from your administrator or start a free trial.
A Working API for Real-Time Data
For this tutorial, we will use the Binance US API to fetch real-time cryptocurrency prices. This API provides market price updates for various trading pairs without requiring authentication. The API endpoint we will be using is:
To ensure that the API is working correctly, you can open the URL in your web browser. If the API is accessible, it will return a JSON response containing the latest price information for multiple cryptocurrency pairs. A sample response from the API looks like this:
[ {"symbol": "BTCUSDT", "price": "23000.00"}, {"symbol": "ETHUSDT", "price": "1620.50"}, {"symbol": "XRPUSDT", "price": "0.2970"}]If you see a similar output, then the API is functional and ready to be integrated with the Fabric Notebook.
Setting Up a Fabric Notebook
A Fabric Notebook will be used to write and execute Python code for fetching data from the API and sending it to Fabric’s EventStream. To create a new Notebook, follow these steps:
Open your Microsoft Fabric workspace.
Click on New Item and select Notebook.
Rename the Notebook to something relevant, such as "Real-Time Crypto Stream".
This Notebook will serve as the foundation for executing the data pipeline that retrieves, formats, and streams real-time data.
Installing Required Python Libraries
Fabric Notebooks support dynamic installation of Python packages, allowing us to install additional libraries as needed. For this implementation, we need three key libraries:
requests: This library is used to make HTTP requests to the API and retrieve the live data.
pytz: This library allows us to manage time zones and convert timestamps, in this blog I want to use Sydney, Australia time.
azure-servicebus: This package enables us to send messages from the Notebook to Fabric’s EventStream.
To install these libraries, open a new code cell in the Fabric Notebook and run the following command:
Once the installation is complete, the Notebook will be ready to execute the necessary Python scripts.
Creating and Configuring an EventStream in Fabric
Fabric’s EventStream is a crucial component for real-time data ingestion. It enables data to be streamed continuously and processed in real time. Before proceeding with the implementation, ensure that an EventStream has been created in your workspace with Fabric License.
Add a custom endpoint as the source into the eventstream, click on publish, and find the key codes to be used in the notebook codes.
Fetch Live Data from an API (Python Code for Running the API)
Now that we have our Fabric Notebook set up and the required Python libraries installed, the next step is to fetch real-time cryptocurrency prices from the Binance API. This section will cover writing a Python script to retrieve data from the API and ensure that the response is structured correctly.
Writing the API Request Code
To retrieve data from the Binance API, we use the requests library to send an HTTP request to the API endpoint. The response will be in JSON format, containing the latest price information for multiple cryptocurrency pairs.
Open a new code cell in your Fabric Notebook and enter the following Python script:
Understanding the Code
The script sends a GET request to the Binance API and retrieves real-time cryptocurrency prices. (You can change this API and use your case's API)
The .raise_for_status() function ensures that an error is raised if the request fails.
The response is converted into JSON format, which allows us to work with the data as Python dictionaries.
Check the output structure, single object or a list and manage them
To verify that the API is working correctly, the script prints the first five records from the response.
Running the Code
Execute the code cell in your Fabric Notebook. If the API is working correctly, the output should look similar to the following:
Sample API Response:[ {"symbol": "BTCUSDT", "price": "23000.00"}, {"symbol": "ETHUSDT", "price": "1620.50"}, {"symbol": "XRPUSDT", "price": "0.2970"}, {"symbol": "ADAUSDT", "price": "0.4692"}, {"symbol": "DOGEUSDT", "price": "0.0621"}]If an error occurs, the script will display an appropriate error message.
Add Date and Time to the Data (Python Code for Timestamps)
Once we have successfully fetched the cryptocurrency prices, we need to add timestamps to each record before sending them to Fabric’s EventStream. This is crucial for tracking when each data point was recorded.
Adding Timestamps to the API Response
In the next code cell of your Fabric Notebook, enter the following script to append date, time, and a formatted datetime value to each record:
Understanding the Code
The script retrieves the current time in Sydney, Australia using the pytz library.
The timestamp is formatted in three different ways:
datetime → Full timestamp in MM/DD/YYYY HH:MM:SS AM/PM format.
date → Date only in DD-MM-YYYY format.
time → Time only in HH:MM:SS format.
The formatted timestamps are added as new columns in each record.
Finally, the updated data structure is printed to verify that the timestamps were added successfully.
Running the Code
Execute the code cell in Fabric. The output should display the first five records, now enriched with date and time fields:
Sample Data with Timestamps:[ {"symbol": "BTCUSDT", "price": "23000.00", "datetime": "04/12/2025 05:00:00 PM", "date": "12-04-2025", "time": "17:00:00"}, {"symbol": "ETHUSDT", "price": "1620.50", "datetime": "04/12/2025 05:00:00 PM", "date": "12-04-2025", "time": "17:00:00"}, {"symbol": "XRPUSDT", "price": "0.2970", "datetime": "04/12/2025 05:00:00 PM", "date": "12-04-2025", "time": "17:00:00"}, {"symbol": "ADAUSDT", "price": "0.4692", "datetime": "04/12/2025 05:00:00 PM", "date": "12-04-2025", "time": "17:00:00"}, {"symbol": "DOGEUSDT", "price": "0.0621", "datetime": "04/12/2025 05:00:00 PM", "date": "12-04-2025", "time": "17:00:00"}]At this stage, we have successfully fetched live data from the Binance API and enriched it with timestamps.
Send Data to EventStream in Fabric (Python Code for EventStream Connection)
The final step is to send the processed data to Microsoft Fabric’s EventStream. This allows Fabric to ingest and process the data in real time.
Sending Data to Fabric EventStream
In a new code cell, enter the following script to connect to Fabric and send data to EventStream:
Understanding the Code
The script connects to Fabric EventStream using the connection string.
It formats each record as a JSON message.
The data is sent in batches for efficient processing.
If the operation is successful, it prints the number of records sent.
Running the Code
Execute the code cell, and if the setup is correct, you should see:
Successfully sent 550 records to EventStream.This confirms that the data is now flowing into Microsoft Fabric in real time.
Automate Data Streaming in Fabric Notebook (Run Every 2 Seconds)
Now that we have successfully fetched data, processed it, and sent it to Fabric’s EventStream, the final step is to automate the script to run continuously. By setting a 2-second interval, we ensure that the notebook fetches new data and sends it to Fabric in real time.
Automating the Notebook Execution
To make the notebook run continuously, we need to loop the execution so that it:
Fetches live data from the API.
Adds timestamps to each record.
Sends the data to Fabric’s EventStream.
Waits for 2 seconds before repeating the process.
To do this, we modify the script to include an infinite loop (while True) with a time.sleep(2) function. This ensures that data is fetched and streamed every 2 seconds without manual intervention.
Below is the full, commented Python script that you can copy and paste into your Fabric Notebook.
How the Code Works
Fetching Live Data from the API
The function fetch_api_data() sends a request to the Binance US API and retrieves real-time cryptocurrency prices.
If the API request fails, an error message is displayed.
Adding Date and Time to Each Record
The function add_timestamps() ensures that each record includes:
datetime → Formatted as MM/DD/YYYY HH:MM:SS AM/PM.
date → Formatted as DD-MM-YYYY.
time → Formatted as HH:MM:SS.
Sending Data to Fabric’s EventStream
The function send_to_eventstream() takes the processed data and sends it to Fabric.
The connection string ensures that messages are properly routed to EventStream.
Each record is converted into JSON before transmission.
Automating the Process Every 2 Seconds
The while loop ensures that the script runs continuously.
After each execution, the script pauses for 2 seconds before fetching new data.
The loop continues until the user manually stops the execution in Fabric Notebook.
Final Thoughts
This Python script provides a fully automated pipeline for fetching API data, processing it, and streaming it into Microsoft Fabric’s Real-Time Intelligence. By setting up a Fabric Notebook, we ensure that data flows continuously into Fabric, enabling real-time analytics and decision-making.
This approach can be applied to various real-time data sources, including:
Stock market prices
IoT sensor readings
Live weather updates
Social media sentiment tracking
Comentários