Azure Monitor Query client library for Python

The Azure Monitor Query client library is used to execute read-only queries against [Azure Monitor][azure_monitor_overview]'s Logs data platform.

Logs - Collects and organizes log and performance data from monitored resources. Data from different sources such as platform logs from Azure services, log and performance data from virtual machines agents, and usage and performance data from apps can be consolidated into a single Azure Log Analytics workspace. The various data types can be analyzed together using the [Kusto Query Language][kusto_query_language].

Important: As of version 2.0.0, MetricsClient and MetricsQueryClient have been removed from the azure-monitor-query package. For metrics querying capabilities, please use the separate azure-monitor-querymetrics package which provides MetricsClient, or the azure-mgmt-monitor package. For more details, see the migration guide.

Resources:

[Source code][source]
[Package (PyPI)][package]
Package (Conda)
[API reference documentation][python-query-ref-docs]
[Service documentation][azure_monitor_overview]
[Samples][samples]
[Change log][changelog]

Getting started

Prerequisites

Python 3.9 or later
An [Azure subscription][azure_subscription]
To query Logs, you need one of the following things:
- An [Azure Log Analytics workspace][azure_monitor_create_using_portal]
- An Azure resource of any kind (Storage Account, Key Vault, Cosmos DB, etc.)

Install the package

Install the Azure Monitor Query client library for Python with [pip][pip]:

pip install azure-monitor-query

Create the client

An authenticated client is required to query Logs. The library includes both synchronous and asynchronous forms of the client. To authenticate, create an instance of a token credential. Use that instance when creating a LogsQueryClient. The following examples use DefaultAzureCredential from the azure-identity package.

Note: For Metrics querying capabilities, please use the separate azure-monitor-querymetrics package which provides MetricsClient, or the azure-mgmt-monitor package.

Synchronous clients

Consider the following example, which creates a synchronous client for Logs querying:

from azure.identity import DefaultAzureCredential
from azure.monitor.query import LogsQueryClient

credential = DefaultAzureCredential()
logs_query_client = LogsQueryClient(credential)

Asynchronous clients

The asynchronous forms of the query client APIs are found in the .aio-suffixed namespace. For example:

from azure.identity.aio import DefaultAzureCredential
from azure.monitor.query.aio import LogsQueryClient

credential = DefaultAzureCredential()
async_logs_query_client = LogsQueryClient(credential)

To use the asynchronous clients, you must also install an async transport, such as aiohttp.

pip install aiohttp

Configure client for Azure sovereign cloud

By default, the client is configured to use the Azure public cloud. To use a sovereign cloud, provide the correct endpoint argument when using LogsQueryClient. For example:

from azure.identity import AzureAuthorityHosts, DefaultAzureCredential
from azure.monitor.query import LogsQueryClient

# Authority can also be set via the AZURE_AUTHORITY_HOST environment variable.
credential = DefaultAzureCredential(authority=AzureAuthorityHosts.AZURE_GOVERNMENT)

logs_query_client = LogsQueryClient(credential, endpoint="https://api.loganalytics.us")

Execute the query

For examples of Logs queries, see the Examples section.

Key concepts

Logs query rate limits and throttling

The Log Analytics service applies throttling when the request rate is too high. Limits, such as the maximum number of rows returned, are also applied on the Kusto queries. For more information, see Query API.

If you're executing a batch logs query, a throttled request returns a LogsQueryError object. That object's code value is ThrottledError.

Logs query

This example shows how to query a Log Analytics workspace. To handle the response and view it in a tabular form, the pandas library is used. See the [samples][samples] if you choose not to use pandas.

Resource-centric logs query

The following example demonstrates how to query logs directly from an Azure resource without the use of a Log Analytics workspace. Here, the query_resource method is used instead of query_workspace. Instead of a workspace ID, an Azure resource identifier is passed in. For example, /subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/{resource-provider}/{resource-type}/{resource-name}.

import os
import pandas as pd
from datetime import timedelta
from azure.monitor.query import LogsQueryClient, LogsQueryStatus
from azure.core.exceptions import HttpResponseError
from azure.identity import DefaultAzureCredential

credential  = DefaultAzureCredential()
client = LogsQueryClient(credential)

query = """AzureActivity | take 5"""

try:
    response = client.query_resource(os.environ['LOGS_RESOURCE_ID'], query, timespan=timedelta(days=1))
    if response.status == LogsQueryStatus.SUCCESS:
        data = response.tables
    else:
        # LogsQueryPartialResult
        error = response.partial_error
        data = response.partial_data
        print(error)

    for table in data:
        df = pd.DataFrame(data=table.rows, columns=table.columns)
        print(df)
except HttpResponseError as err:
    print("something fatal happened")
    print(err)

Specify timespan

The timespan parameter specifies the time duration for which to query the data. This value can take one of the following forms:

a timedelta
a timedelta and a start datetime
a start datetime/end datetime

For example:

import os
import pandas as pd
from datetime import datetime, timezone
from azure.monitor.query import LogsQueryClient, LogsQueryResult
from azure.identity import DefaultAzureCredential
from azure.core.exceptions import HttpResponseError

credential = DefaultAzureCredential()
client = LogsQueryClient(credential)

query = """AppRequests | take 5"""

start_time=datetime(2021, 7, 2, tzinfo=timezone.utc)
end_time=datetime(2021, 7, 4, tzinfo=timezone.utc)

try:
    response = client.query_workspace(
        workspace_id=os.environ['LOG_WORKSPACE_ID'],
        query=query,
        timespan=(start_time, end_time)
        )
    if response.status == LogsQueryStatus.SUCCESS:
        data = response.tables
    else:
        # LogsQueryPartialResult
        error = response.partial_error
        data = response.partial_data
        print(error)

    for table in data:
        df = pd.DataFrame(data=table.rows, columns=table.columns)
        print(df)
except HttpResponseError as err:
    print("something fatal happened")
    print(err)

Handle logs query response

The query_workspace API returns either a LogsQueryResult or a LogsQueryPartialResult object. The batch_query API returns a list that can contain LogsQueryResult, LogsQueryPartialResult, and LogsQueryError objects. Here's a hierarchy of the response:

LogsQueryResult
|---statistics
|---visualization
|---tables (list of `LogsTable` objects)
    |---name
    |---rows
    |---columns
    |---columns_types

LogsQueryPartialResult
|---statistics
|---visualization
|---partial_error (a `LogsQueryError` object)
    |---code
    |---message
    |---details
    |---status
|---partial_data (list of `LogsTable` objects)
    |---name
    |---rows
    |---columns
    |---columns_types

The LogsQueryResult directly iterates over the table as a convenience. For example, to handle a logs query response with tables and display it using pandas:

response = client.query(...)
for table in response:
    df = pd.DataFrame(table.rows, columns=[col.name for col in table.columns])

A full sample can be found here.

In a similar fashion, to handle a batch logs query response:

for result in response:
    if result.status == LogsQueryStatus.SUCCESS:
        for table in result:
            df = pd.DataFrame(table.rows, columns=table.columns)
            print(df)