sqlalchemy-redshift
Amazon Redshift Dialect for sqlalchemy
Rank: #1869Downloads: 4,445,355 (30 days)Stars: 223Forks: 163
Description
sqlalchemy-redshift
===================
Amazon Redshift dialect for SQLAlchemy.
Installation
------------
The package is available on PyPI::
pip install sqlalchemy-redshift
.. warning::
This dialect requires either ``redshift_connector`` or ``psycopg2``
to work properly. It does not provide
it as required, but relies on you to select the distribution you need:
* psycopg2 - standard distribution of psycopg2, requires compilation so few system dependencies are required for it
* psycopg2-binary - already compiled distribution (no system dependencies are required)
* psycopg2cffi - pypy compatible version
See `Psycopg2's binary install docs <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>`_
for more context on choosing a distribution.
Usage
-----
The DSN format is similar to that of regular Postgres::
>>> import sqlalchemy as sa
>>> sa.create_engine('redshift+psycopg2://username@host.amazonaws.com:5439/database')
Engine(redshift+psycopg2://username@host.amazonaws.com:5439/database)
See the `RedshiftDDLCompiler documentation
<https://sqlalchemy-redshift.readthedocs.org/en/latest/ddl-compiler.html>`_
for details on Redshift-specific features the dialect supports.
Running Tests
-------------
Tests are ran via tox and can be run with the following command::
$ tox
However, this will not run integration tests unless the following
environment variables are set:
* REDSHIFT_HOST
* REDSHIFT_PORT
* REDSHIFT_USERNAME
* PGPASSWORD (this is the redshift instance password)
* REDSHIFT_DATABASE
* REDSHIFT_IAM_ROLE_ARN
Note that the IAM role specified will need to be associated with
redshift cluster and have the correct permissions to create databases
and tables as well drop them. Exporting these environment variables in
your shell and running ``tox`` will run the integration tests against
a real redshift instance. Practice caution when running these tests
against a production instance.
Continuous Integration (CI)
---------------------------
Project CI is built using AWS CodePipeline and CloudFormation. Please see the ``ci/`` folder and included ``README.txt``
for details on how to spin up the project's CI.
Releasing
---------
To perform a release, you will need to be an admin for the project on
GitHub and on PyPI. Contact the maintainers if you need that access.
You will need to have a `~/.pypirc` with your PyPI credentials and
also the following settings::
[zest.releaser]
create-wheels = yes
To perform a release, run the following::
python -m venv ~/.virtualenvs/dist
workon dist
pip install -U pip setuptools wheel
pip install -U tox zest.releaser
fullrelease # follow prompts, use semver ish with versions.
The releaser will handle updating version data on the package and in
CHANGES.rst along with tagging the repo and uploading to PyPI.
0.8.14 (2023-04-07)
-------------------
- Override new upstream postgres method that fails against redshift (`Pull #266 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/266>`_)
- Fix table reflection broken for non-superusers
(`Pull #276 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/276>`_)
- Fix Broken Reflection for 1.4 FutureEngine
(`Pull #277 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/277>`_)
0.8.13 (2023-03-28)
-------------------
- Add spectrum support
(`Pull #263 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/263>`_)
- Drop support for Python 3.5
0.8.12 (2022-12-08)
-------------------
- Fix SQLAlchemy's "supports_statement_cache" (`Pull #259 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/259>`_)
0.8.11 (2022-07-27)
-------------------
- Disable redshift_connector dialect statement cache (`Pull #257 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/257>`_)
0.8.10 (2022-07-21)
-------------------
- Support HLLSKETCH Redshift datatypes
(`Pull #246 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/246>`_)
- Disable supports_statement_cache
(`Pull #249 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/249>`_)
- Fix doc, lint CI dependency issues
(`Pull #250 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/250>`_)
- Fix redshift_connector dialect column encoding
(`Pull #255 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/255>`_)
0.8.9 (2021-12-15)
------------------
- Support inspection of Redshift datatypes
(`Pull #242 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/242>`_)
0.8.8 (2021-11-03)
------------------
- Remove support for Python 2.7; now requires python ``>=3.4``
(`Pull #234 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/234>`_)
- Support GEOMETRY, SUPER Redshift datatypes
(`Pull #235 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/235>`_)
0.8.7 (2021-10-27)
------------------
- Initial SQLAlchemy 2.0.x support
(`Pull #237 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/237>`_)
0.8.6 (2021-09-22)
------------------
- Add RedshiftDialect_redshift_connector
(`Pull #232 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/232>`_)
- Create RedshiftDialectMixin class. Add RedshiftDialect_psycopg2cffi.
(`Pull #231 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/231>`_)
0.8.5 (2021-08-23)
------------------
- Support TIMETZ datatype
(`Pull #229 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/229>`_)
- Fix RelationKey unquoted issue
(`Pull #228 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/228>`_)
0.8.4 (2021-07-15)
------------------
- Improve reflection performance by fetching/caching metadata per schema
rather than for the entire database
(`Pull #223 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/223>`_)
0.8.3 (2021-07-07)
------------------
- SQLAlchemy 1.4.x support
(`Pull #221 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/221>`_)
0.8.2 (2021-01-08)
------------------
- Allow supplying multiple role ARNs in COPY and UNLOAD commands. This allows
the first role to assume other roles as explained
`here <https://docs.aws.amazon.com/redshift/latest/mgmt/authorizing-redshift-service.html#authorizing-redshift-service-chaining-roles>`_.
0.8.1 (2020-07-15)
------------------
- Support AWS partitions for role-based access control in COPY and UNLOAD
commands. This allows these commands to be used, e.g. in GovCloud.
0.8.0 (2020-06-30)
------------------
- Add option to drop materialized view with CASCADE
(`Pull #204 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/204>`_)
- Fix invalid SQLAlchemy version comparison
(`Pull #206 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/206>`_)
0.7.9 (2020-05-29)
------------------
- Fix for supporting SQLAlchemy 1.3.11+
(`Issue #195 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/195>`_)
0.7.8 (2020-05-27)
------------------
- Added support for materialized views
(`Issue #202 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/202>`_)
- Fix reflection of unique constraints
(`Issue #199 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/199>`_)
- Support for altering column comments in Alembic migrations
(`Issue #191 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/191>`_)
0.7.7 (2020-02-02)
------------------
- Import Iterable from collections.abc for Python 3.9 compatibility
(`Issue #189 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/189>`_)
- Add support for Parquet format in ``UNLOAD`` command
(`Issue #187 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/187>`_)
0.7.6 (2020-01-17)
------------------
- Fix unhashable type error for sortkey reflection in SQLAlchemy >= 1.3.11
(`Issue #180 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/180>`_)
- Expose supported types for import from the dialect
(`Issue #181 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/181>`_)
- Reflect column comments
(`Issue #186 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/186>`_)
0.7.5 (2019-10-09)
------------------
- Extend psycopg2 package version check to also support psycopg2-binary and psycopg2cffi
(`Issue #178 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/178>`_)
0.7.4 (2019-10-08)
------------------
- Drop hard dependency on psycopg2 but require package to be present on runtime
(`Issue #165 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/165>`_)
- Switch from info to keyword arguments on columns for ``SQLAlchemy >= 1.3.0``
(`Issue #161 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/161>`_)
- Add support for column info on redshift late binding views
(`Issue #159 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/159>`_)
- Add support for ``MAXFILESIZE`` argument to ``UNLOAD``.
(`Issue #123 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/123>`_)
- Add support for the `CREATE LIBRARY`_ command.
(`Issue #124 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/124>`_)
- Add support for the `ALTER TABLE APPEND`_ command.
(`Issue #162 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/pull/162>`_)
- Add support for the ``CSV`` format to `UnloadFromSelect`.
(`Issue #169 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/169>`_)
- Update the list of reserved words (adds "az64" and "language")
(`Issue #176 <https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/176>`_)
.. _CREATE LIBRARY: https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_LIBRARY.html
.. _ALTER TABLE APPEND: https://docs.aws.amazon.com/redshift/latest/dg/r_ALTER_TABLE_APPEND.html
0.7.3 (2019-01-16)