A lightweight, extensible library that simplifies interaction with heterogeneous data systems. It standardises connections, authentication, and data transfer into consistent, Pandas‑friendly workflows, enabling seamless integration across databases and APIs without the overhead of system‑specific code.
Report Bug
·
Request Feature
Table of Contents
This library provides a unified interface for interacting with diverse data systems, eliminating the need to manage the intricacies of individual platforms. By abstracting connection handling, authentication, data retrieval, and writing operations into simple, consistent functions, it enables developers and data engineers to focus on analysis and integration rather than boilerplate code.
Whether working with traditional databases or modern API endpoints, the library standardises workflows into familiar patterns such as reading into Pandas DataFrames, chunked processing for large datasets, and seamless write‑back operations. Built with extensibility in mind, new data systems can be added without disrupting existing usage, ensuring the library evolves alongside your environment.
Designed for clarity, maintainability, and scalability, this library is a practical foundation for projects that demand reliable access to heterogeneous data sources.
pip install cnxnsThe dbms module provides a streamlined interface for interacting with SQL-based data systems through SQLAlchemy. It abstracts connection setup, data ingestion, and write-back operations into concise, reusable functions that integrate seamlessly with Pandas. With built-in support for chunked reading and writing, it enables efficient handling of large datasets while maintaining clarity and consistency across varied database backends.
- Microsoft SQL Server
- MySQL/MariaDB
- Either:
- ODBC Driver X for SQL Server (tested with 18)
- MySQL ODBC X driver (tested with 9.4)
from cnxns import dbms as db
e = db.dbms_cnxn(
dbms = "mssql",
server = "localhost",
uid = "sa",
pwd = "YourStrong@Passw0rd",
database = "dev",
)
df = db.dbms_reader(
e,
query = "SELECT TOP(1000) * FROM myAwesomeTable",
)
print(df)
for chunk in db.dbms_read_chunks(
e,
table_name = "myAwesomeTable",
chunksize = 1000,
):
db.dbms_writer(
e,
df,
"myAwesomeTableSnapshot",
if_exists="append",
)The m365 module abstracts authentication and data access workflows for Microsoft 365 APIs, streamlining interactions with services like Graph and Dataverse. It handles token generation via MSAL and Azure AD app registration, enabling secure, reusable access patterns. With built-in support for chunked queries and JSON response handling, it simplifies integration into data pipelines and analytical workflows while maintaining flexibility across varied Microsoft 365 endpoints.
- Dataverse REST API
- Microsoft 365 Graph API
- An application registered in Azure Active Directory. You will need:
- Tennant ID
- Client ID
- Client Secret
from cnxns.api import m365
result = m365.query_api(
client_id="A123",
client_secret="B234",
tenant_id="C345",
base_url="https://myamazingcompany.crm.dynamics.com",
api_url="api/data/v9.0",
query="/accounts?$select=accountid,versionnumber",
chunksize=1000,
)Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See LICENSE.txt for more information.