Expand joins when calculating PostgreSQL "WITH FOR UPDATE OF"

Modified the :paramref:`.Select.with_for_update.of` parameter so that if a join or other composed selectable is passed, the individual :class:`.Table` objects will be filtered from it, allowing one to pass a join() object to the parameter, as occurs normally when using joined table inheritance with the ORM. Pull request courtesy Raymond Lu. Fixes: #4550 Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #4551 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4551 Pull-request-sha: 452da77d154a4087d530456db1c9af207d65cef4 Change-Id: If4b7c231f7b71190d7245543959fb5c3351125a1
Changed urllib.quote_plus to urllib.parse.quote_plus (Python 3 versio…
2019-03-21 10:25:29 -04:00 · 2019-03-18 10:51:53 -04:00 · 2019-03-16 17:42:59 -04:00 · 2019-03-11 14:49:23 +00:00 · 2019-03-10 23:44:08 -04:00 · 2019-03-09 14:38:14 -05:00
90 changed files with 1969 additions and 385 deletions
@@ -25,4 +25,8 @@ sqlnet.log
 /test?.py
 /test.py
 /mapping_setup.py
-*.rej
+*.rej
+test/test_schema.db
+*test_schema.db
+.idea
+/Pipfile*
@@ -11,9 +11,73 @@
        :start-line: 5

 .. changelog::
-    :version: 1.2.18
+    :version: 1.2.19
    :include_notes_from: unreleased_12

+.. changelog::
+    :version: 1.2.18
+    :released: February 15, 2019
+
+    .. change::
+       :tags: bug, orm
+       :tickets: 4468
+
+       Fixed a regression in 1.2 where a wildcard/load_only loader option would
+       not work correctly against a loader path where of_type() were used to limit
+       to a particular subclass.  The fix only works for of_type() of a simple
+       subclass so far, not a with_polymorphic entity which will be addressed in a
+       separate issue; it is unlikely this latter case was working previously.
+
+
+    .. change::
+       :tags: bug, orm
+       :tickets: 4489
+
+       Fixed fairly simple but critical issue where the
+       :meth:`.SessionEvents.pending_to_persistent` event would be invoked for
+       objects not just when they move from pending to persistent, but when they
+       were also already persistent and just being updated, thus causing the event
+       to be invoked for all objects on every update.
+
+    .. change::
+       :tags: bug, sql
+       :tickets: 4485
+
+       Fixed issue where the :class:`.JSON` type had a read-only
+       :attr:`.JSON.should_evaluate_none` attribute, which would cause failures
+       when making use of the :meth:`.TypeEngine.evaluates_none` method in
+       conjunction with this type.  Pull request courtesy Sanjana S.
+
+    .. change::
+       :tags: bug, mssql
+       :tickets: 4499
+
+       Fixed bug where the SQL Server "IDENTITY_INSERT" logic that allows an INSERT
+       to proceed with an explicit value on an IDENTITY column was not detecting
+       the case where :meth:`.Insert.values` were used with a dictionary that
+       contained a :class:`.Column` as key and a SQL expression as a value.
+
+    .. change::
+       :tags: bug, sqlite
+       :tickets: 4474
+
+       Fixed bug in SQLite DDL where using an expression as a server side default
+       required that it be contained within parenthesis to be accepted by the
+       sqlite parser.  Pull request courtesy Bartlomiej Biernacki.
+
+    .. change::
+       :tags: bug, mysql
+       :tickets: 4492
+
+       Fixed a second regression caused by :ticket:`4344` (the first was
+       :ticket:`4361`), which works around MySQL issue 88718, where the lower
+       casing function used was not correct for Python 2 with OSX/Windows casing
+       conventions, which would then raise ``TypeError``.  Full coverage has been
+       added to this logic so that every codepath is exercised in a mock style for
+       all three casing conventions on all versions of Python. MySQL 8.0 has
+       meanwhile fixed issue 88718 so the workaround is only applies to a
+       particular span of MySQL 8.0 versions.
+
 .. changelog::
    :version: 1.2.17
    :released: January 25, 2019
@@ -10,8 +10,95 @@
    .. include:: changelog_11.rst
        :start-line: 5

+.. changelog::
+    :version: 1.3.2
+    :include_notes_from: unreleased_13
+
+.. changelog::
+    :version: 1.3.1
+    :released: March 9, 2019
+
+    .. change::
+       :tags: bug, mssql
+       :tickets: 4525
+
+       Fixed regression in SQL Server reflection due to :ticket:`4393` where the
+       removal of open-ended ``**kw`` from the :class:`.Float` datatype caused
+       reflection of this type to fail due to a "scale" argument being passed.
+
+    .. change::
+       :tags: bug, orm, ext
+       :tickets: 4522
+
+       Fixed regression where an association proxy linked to a synonym would no
+       longer work, both at instance level and at class level.
+
+.. changelog::
+    :version: 1.3.0
+    :released: March 4, 2019
+
+    .. change::
+       :tags: feature, schema
+       :tickets: 4517
+
+       Added new parameters :paramref:`.Table.resolve_fks` and
+       :paramref:`.MetaData.reflect.resolve_fks` which when set to False will
+       disable the automatic reflection of related tables encountered in
+       :class:`.ForeignKey` objects, which can both reduce SQL overhead for omitted
+       tables as well as avoid tables that can't be reflected for database-specific
+       reasons.  Two :class:`.Table` objects present in the same :class:`.MetaData`
+       collection can still refer to each other even if the reflection of the two
+       tables occurred separately.
+
+
+    .. change::
+       :tags: feature, orm
+       :tickets: 4316
+
+       The :meth:`.Query.get` method can now accept a dictionary of attribute keys
+       and values as a means of indicating the primary key value to load; is
+       particularly useful for composite primary keys.  Pull request courtesy
+       Sanjana S.
+
+    .. change::
+       :tags: feature, orm
+       :tickets: 3133
+
+       A SQL expression can now be assigned to a primary key attribute for an ORM
+       flush in the same manner as ordinary attributes as described in
+       :ref:`flush_embedded_sql_expressions` where the expression will be evaulated
+       and then returned to the ORM using RETURNING, or in the case of pysqlite,
+       works using the cursor.lastrowid attribute.Requires either a database that
+       supports RETURNING (e.g. Postgresql, Oracle, SQL Server) or pysqlite.
+
+    .. change::
+       :tags: bug, sql
+       :tickets: 4509
+
+       The :class:`.Alias` class and related subclasses :class:`.CTE`,
+       :class:`.Lateral` and :class:`.TableSample` have been reworked so that it is
+       not possible for a user to construct the objects directly.  These constructs
+       require that the standalone construction function or selectable-bound method
+       be used to instantiate new objects.
+
+
+    .. change::
+       :tags: feature, engine
+       :tickets: 4500
+
+       Revised the formatting for :class:`.StatementError` when stringified. Each
+       error detail is broken up over multiple newlines instead of spaced out on a
+       single line.  Additionally, the SQL representation now stringifies the SQL
+       statement rather than using ``repr()``, so that newlines are rendered as is.
+       Pull request courtesy Nate Clark.
+
+       .. seealso::
+
+            :ref:`change_4500`
+
 .. changelog::
    :version: 1.3.0b3
+    :released: March 4, 2019
    :released: February 8, 2019

    .. change::
@@ -115,6 +202,7 @@

 .. changelog::
    :version: 1.3.0b2
+    :released: March 4, 2019
    :released: January 25, 2019

    .. change::
@@ -342,6 +430,7 @@

 .. changelog::
    :version: 1.3.0b1
+    :released: March 4, 2019
    :released: November 16, 2018

    .. change::
@@ -1059,6 +1059,54 @@ The above suffix ``a79e`` is based on the md5 hash of the long name and will
 generate the same value every time to produce consistent names for a given
 schema.

+Note that the truncation logic also raises :class:`.IdentifierError` when a
+constraint name is explicitly too large for a given dialect.  This has been
+the behavior for an :class:`.Index` object for a long time, but is now applied
+to other kinds of constraints as well::
+
+    from sqlalchemy import Column
+    from sqlalchemy import Integer
+    from sqlalchemy import MetaData
+    from sqlalchemy import Table
+    from sqlalchemy import UniqueConstraint
+    from sqlalchemy.dialects import postgresql
+    from sqlalchemy.schema import AddConstraint
+
+    m = MetaData()
+    t = Table("t", m, Column("x", Integer))
+    uq = UniqueConstraint(
+        t.c.x,
+        name="this_is_too_long_of_a_name_for_any_database_backend_even_postgresql",
+    )
+
+    print(AddConstraint(uq).compile(dialect=postgresql.dialect()))
+
+will output::
+
+    sqlalchemy.exc.IdentifierError: Identifier
+    'this_is_too_long_of_a_name_for_any_database_backend_even_postgresql'
+    exceeds maximum length of 63 characters
+
+The exception raise prevents the production of non-deterministic constraint
+names truncated by the database backend which are then not compatible with
+database migrations later on.
+
+To apply SQLAlchemy-side truncation rules to the above identifier, use the
+:func:`.conv` construct::
+
+    uq = UniqueConstraint(
+        t.c.x,
+        name=conv("this_is_too_long_of_a_name_for_any_database_backend_even_postgresql"),
+    )
+
+This will again output deterministically truncated SQL as in::
+
+    ALTER TABLE t ADD CONSTRAINT this_is_too_long_of_a_name_for_any_database_backend_eve_ac05 UNIQUE (x)
+
+There is not at the moment an option to have the names pass through to allow
+database-side truncation.  This has already been the case for :class:`.Index`
+names for some time and issues have not been raised.
+
 The change also repairs two other issues.  One is that the  ``column_0_key``
 token wasn't available even though this token was documented, the other was
 that the ``referred_column_0_name`` token would  inadvertently render the
@@ -1662,3 +1710,33 @@ primary key column::
 :ticket:`4362`

 :ticket:`4235`
+
+.. _change_4500:
+
+Changed StatementError formatting (newlines and %s)
+=================================================================================
+
+Two changes are introduced to the string representation for ``StatementError``.
+The "detail" and "SQL" portions of the string representation are now
+separated by newlines, and newlines that are present in the original SQL
+statement are maintained.   The goal is to improve readability while still
+keeping the original error message on one line for logging purposes.
+
+This means that an error message that previously looked like this::
+
+    sqlalchemy.exc.StatementError: (sqlalchemy.exc.InvalidRequestError) A value is required for bind parameter 'id' [SQL: 'select * from reviews\nwhere id = ?'] (Background on this error at: http://sqlalche.me/e/cd3x)
+
+Will now look like this::
+
+    sqlalchemy.exc.StatementError: (sqlalchemy.exc.InvalidRequestError) A value is required for bind parameter 'id'
+    [SQL: select * from reviews
+    where id = ?]
+    (Background on this error at: http://sqlalche.me/e/cd3x)
+
+The primary impact of this change is that consumers can no longer assume that
+a complete exception message is on a single line, however the original
+"error" portion that is generated from the DBAPI driver or SQLAlchemy internals
+will still be on the first line.
+
+:ticket:`4500`
+
@@ -0,0 +1,11 @@
+.. change::
+   :tags: bug, engine
+   :tickets: 4406
+
+   Comparing two objects of :class:`.URL` using ``__eq__()`` did not take port
+   number into consideration, two objects differing only by port number were
+   considered equal. Port comparison is now added in ``__eq__()`` method of
+   :class:`.URL`, objects differing by port number are now not equal.
+   Additionally, ``__ne__()`` was not implemented for :class:`.URL` which
+   caused unexpected result when ``!=`` was used in Python2, since there are no
+   implied relationships among the comparison operators in Python2.
@@ -1,10 +0,0 @@
-.. change::
-   :tags: bug, orm
-   :tickets: 4468
-
-    Fixed a regression in 1.2 where a wildcard/load_only loader option would
-    not work correctly against a loader path where of_type() were used to limit
-    to a particular subclass.  The fix only works for of_type() of a simple
-    subclass so far, not a with_polymorphic entity which will be addressed in a
-    separate issue; it is unlikely this latter case was working previously.
-
@@ -1,7 +0,0 @@
-.. change::
-   :tags: bug, sqlite
-   :tickets: 4474
-
-   Fixed bug in SQLite DDL where using an expression as a server side default
-   required that it be contained within parenthesis to be accepted by the
-   sqlite parser.  Pull request courtesy Bartlomiej Biernacki.
@@ -0,0 +1,8 @@
+.. change::
+   :tags: bug, oracle
+   :tickets: 4506
+
+   Added support for reflection of the :class:`.NCHAR` datatype to the Oracle
+   dialect, and added :class:`.NCHAR` to the list of types exported by the
+   Oracle dialect.
+
@@ -0,0 +1,13 @@
+.. change::
+   :tags: bug, orm
+   :tickets: 4507
+
+   Fixed a regression in 1.2 due to the introduction of baked queries for
+   relationship lazy loaders, where a race condition is created during the
+   generation of the "lazy clause" which occurs within a memoized attribute. If
+   two threads initialize the memoized attribute concurrently, the baked query
+   could be generated with bind parameter keys that are then replaced with new
+   keys by the next run, leading to a lazy load query that specifies the
+   related criteria as ``None``. The fix establishes that the parameter names
+   are fixed before the new clause and parameter objects are generated, so that
+   the names are the same every time.
@@ -0,0 +1,7 @@
+.. change::
+   :tags: bug, examples
+   :tickets: 4528
+
+   Fixed bug in large_resultsets example case where a re-named "id" variable
+   due to code reformatting caused the test to fail.  Pull request courtesy
+   Matt Schuchhardt.
@@ -0,0 +1,8 @@
+.. change::
+   :tags: bug, mssql
+   :tickets: 4536
+   :versions: 1.3.1
+
+   A commit() is emitted after an isolation level change to SNAPSHOT, as both
+   pyodbc and pymssql open an implicit transaction which blocks subsequent SQL
+   from being emitted in the current transaction.
@@ -0,0 +1,13 @@
+.. change::
+   :tags: bug, orm
+   :tickets: 4537
+
+   Fixed bug where use of :func:`.with_polymorphic` or other aliased construct
+   would not properly adapt when the aliased target were used as the
+   :meth:`.Select.correlate_except` target of a subquery used inside of a
+   :func:`.column_property`. This required a fix to the clause adaption
+   mechanics to properly handle a selectable that shows up in the "correlate
+   except" list, in a similar manner as which occurs for selectables that show
+   up in the "correlate" list.  This is ultimately a fairly fundamental bug
+   that has lasted for a long time but it is hard to come across it.
+
@@ -0,0 +1,10 @@
+.. change::
+   :tags: bug, postgresql
+   :tickets: 4550
+
+   Modified the :paramref:`.Select.with_for_update.of` parameter so that if a
+   join or other composed selectable is passed, the individual :class:`.Table`
+   objects will be filtered from it, allowing one to pass a join() object to
+   the parameter, as occurs normally when using joined table inheritance with
+   the ORM.  Pull request courtesy Raymond Lu.
+
@@ -109,9 +109,9 @@ copyright = u'2007-2019, the SQLAlchemy authors and contributors'
 # The short X.Y version.
 version = "1.3"
 # The full version, including alpha/beta/rc tags.
-release = "1.3.0b3"
+release = "1.3.1"

-release_date = "February 8, 2019"
+release_date = "March 9, 2019"

 site_base = os.environ.get("RTD_SITE_BASE", "http://www.sqlalchemy.org")
 site_adapter_template = "docs_adapter.mako"
@@ -450,12 +450,12 @@ connections from the pool so that it makes all new ones.  Below is
 a simple version using ``multiprocessing.Process``, but this idea
 should be adapted to the style of forking in use::

-    eng = create_engine("...")
+    engine = create_engine("...")

    def run_in_process():
-      eng.dispose()
+      engine.dispose()

-      with eng.connect() as conn:
+      with engine.connect() as conn:
          conn.execute("...")

    p = Process(target=run_in_process)
@@ -468,7 +468,7 @@ This is a little more magical but probably more foolproof::
    from sqlalchemy import exc
    import os

-    eng = create_engine("...")
+    engine = create_engine("...")

    @event.listens_for(engine, "connect")
    def connect(dbapi_connection, connection_record):
@@ -33,6 +33,7 @@ Currently maintained external dialect projects for SQLAlchemy include:


 * `ibm_db_sa <http://code.google.com/p/ibm-db/wiki/README>`_ - driver for IBM DB2 and Informix.
+* `PyHive <https://github.com/dropbox/PyHive#sqlalchemy>`_ - driver for `Apache Hive <https://hive.apache.org/>`_ and `Presto <https://prestodb.github.io/>`_.
 * `sqlalchemy-redshift <https://pypi.python.org/pypi/sqlalchemy-redshift>`_ - driver for Amazon Redshift, adapts
  the existing PostgreSQL/psycopg2 driver.
 * `sqlalchemy-drill <https://github.com/JohnOmernik/sqlalchemy-drill>`_ - driver for Apache Drill.
@@ -45,4 +46,3 @@ Currently maintained external dialect projects for SQLAlchemy include:
 * `sqlalchemy-tds <https://github.com/m32/sqlalchemy-tds>`_ - driver for MS-SQL,
  on top of `python-tds <https://github.com/denisenkom/pytds>`_.
 * `crate <https://github.com/crate/crate-python>`_ - driver for `CrateDB <https://crate.io/>`_.
-
@@ -14,10 +14,13 @@ they originate from :mod:`sqlalchemy.types` or from the local dialect::

    from sqlalchemy.dialects.oracle import \
                BFILE, BLOB, CHAR, CLOB, DATE, \
-                DOUBLE_PRECISION, FLOAT, INTERVAL, LONG, NCLOB, \
+                DOUBLE_PRECISION, FLOAT, INTERVAL, LONG, NCLOB, NCHAR, \
                NUMBER, NVARCHAR, NVARCHAR2, RAW, TIMESTAMP, VARCHAR, \
                VARCHAR2

+.. versionadded:: 1.2.19 Added :class:`.NCHAR` to the list of datatypes
+   exported by the Oracle dialect.
+
 Types which are specific to Oracle, or have Oracle-specific
 construction arguments, are as follows:

@@ -494,6 +494,59 @@ be used for querying::
            join(uka, User.keywords.local_attr).\
            join(ka, User.keywords.remote_attr)

+.. _cascade_scalar_deletes:
+
+Cascading Scalar Deletes
+------------------------
+
+.. versionadded:: 1.3
+
+Given a mapping as::
+
+    class A(Base):
+        __tablename__ = 'test_a'
+        id = Column(Integer, primary_key=True)
+        ab = relationship(
+            'AB', backref='a', uselist=False)
+        b = association_proxy(
+            'ab', 'b', creator=lambda b: AB(b=b),
+            cascade_scalar_deletes=True)
+
+
+    class B(Base):
+        __tablename__ = 'test_b'
+        id = Column(Integer, primary_key=True)
+        ab = relationship('AB', backref='b', cascade='all, delete-orphan')
+
+
+    class AB(Base):
+        __tablename__ = 'test_ab'
+        a_id = Column(Integer, ForeignKey(A.id), primary_key=True)
+        b_id = Column(Integer, ForeignKey(B.id), primary_key=True)
+
+An assigment to ``A.b`` will generate an ``AB`` object::
+
+    a.b = B()
+
+The ``A.b`` association is scalar, and includes use of the flag
+:paramref:`.AssociationProxy.cascade_scalar_deletes`.  When set, setting ``A.b``
+to ``None`` will remove ``A.ab`` as well::
+
+    a.b = None
+    assert a.ab is None
+
+When :paramref:`.AssociationProxy.cascade_scalar_deletes` is not set,
+the association object ``a.ab`` above would remain in place.
+
+Note that this is not the behavior for collection-based association proxies;
+in that case, the intermediary association object is always removed when
+members of the proxied collection are removed.  Whether or not the row is
+deleted depends on the relationship cascade setting.
+
+.. seealso::
+
+    :ref:`unitofwork_cascades`
+
 API Documentation
 -----------------

@@ -32,6 +32,44 @@ flush/commit operation, the ``value`` attribute on ``someobject`` above is
 expired, so that when next accessed the newly generated value will be loaded
 from the database.

+The feature also has conditional support to work in conjunction with
+primary key columns.  A database that supports RETURNING, e.g. PostgreSQL,
+Oracle, or SQL Server, or as a special case when using SQLite with the pysqlite
+driver and a single auto-increment column, a SQL expression may be assigned
+to a primary key column as well.  This allows both the SQL expression to
+be evaluated, as well as allows any server side triggers that modify the
+primary key value on INSERT, to be successfully retrieved by the ORM as
+part of the object's primary key::
+
+
+    class Foo(Base):
+        __tablename__ = 'foo'
+        pk = Column(Integer, primary_key=True)
+        bar = Column(Integer)
+
+    e = create_engine("postgresql://scott:tiger@localhost/test", echo=True)
+    Base.metadata.create_all(e)
+
+    session = Session(e)
+
+    foo = Foo(pk=sql.select([sql.func.coalesce(sql.func.max(Foo.pk) + 1, 1)])
+    session.add(foo)
+    session.commit()
+
+On PostgreSQL, the above :class:`.Session` will emit the following INSERT:
+
+.. sourcecode:: sql
+
+    INSERT INTO foo (foopk, bar) VALUES
+    ((SELECT coalesce(max(foo.foopk) + %(max_1)s, %(coalesce_2)s) AS coalesce_1
+    FROM foo), %(bar)s) RETURNING foo.foopk
+
+.. versionadded:: 1.3
+    SQL expressions can now be passed to a primary key column during an ORM
+    flush; if the database supports RETURNING, or if pysqlite is in use, the
+    ORM will be able to retrieve the server-generated value as the value
+    of the primary key attribute.
+
 .. _session_sql_expressions:

 Using SQL Expressions with Sessions
@@ -714,8 +714,11 @@ Additionally, autoflush can be temporarily disabled by setting the
    mysession = Session()
    mysession.autoflush = False

-Some autoflush-disable recipes are available at `DisableAutoFlush
-<http://www.sqlalchemy.org/trac/wiki/UsageRecipes/DisableAutoflush>`_.
+More conveniently, it can be turned off within a context managed block using :attr:`.Session.no_autoflush`::
+
+    with mysession.no_autoflush:
+        mysession.add(some_object)
+        mysession.flush()

 The flush process *always* occurs within a transaction, even if the
 :class:`~sqlalchemy.orm.session.Session` has been configured with
@@ -170,7 +170,7 @@ def _test_dbapi_raw(n, make_objects):
        # ORM land anyway :)
        class SimpleCustomer(object):
            def __init__(self, id_, name, description):
-                self.id = id_
+                self.id_ = id_
                self.name = name
                self.description = description

@@ -184,7 +184,7 @@ def _test_dbapi_raw(n, make_objects):
        for row in cursor.fetchall():
            # ensure that we fully fetch!
            customer = SimpleCustomer(
-                id=row[0], name=row[1], description=row[2]
+                id_=row[0], name=row[1], description=row[2]
            )
    else:
        for row in cursor.fetchall():
@@ -122,7 +122,7 @@ from .engine import create_engine  # noqa nosort
 from .engine import engine_from_config  # noqa nosort


-__version__ = "1.3.0b3"
+__version__ = '1.3.2'


 def __go(lcls):
@@ -876,8 +876,10 @@ class REAL(sqltypes.REAL):
    __visit_name__ = "REAL"

    def __init__(self, **kw):
-        # REAL is a synonym for FLOAT(24) on SQL server
-        kw["precision"] = 24
+        # REAL is a synonym for FLOAT(24) on SQL server.
+        # it is only accepted as the word "REAL" in DDL, the numeric
+        # precision value is not allowed to be present
+        kw.setdefault("precision", 24)
        super(REAL, self).__init__(**kw)


@@ -1380,13 +1382,21 @@ class MSExecutionContext(default.DefaultExecutionContext):
                    and (
                        (
                            self.compiled.statement._has_multi_parameters
-                            and seq_column.key
-                            in self.compiled.statement.parameters[0]
+                            and (
+                                seq_column.key
+                                in self.compiled.statement.parameters[0]
+                                or seq_column
+                                in self.compiled.statement.parameters[0]
+                            )
                        )
                        or (
                            not self.compiled.statement._has_multi_parameters
-                            and seq_column.key
-                            in self.compiled.statement.parameters
+                            and (
+                                seq_column.key
+                                in self.compiled.statement.parameters
+                                or seq_column
+                                in self.compiled.statement.parameters
+                            )
                        )
                    )
                )
@@ -2225,6 +2235,8 @@ class MSDialect(default.DefaultDialect):
        cursor = connection.cursor()
        cursor.execute("SET TRANSACTION ISOLATION LEVEL %s" % level)
        cursor.close()
+        if level == "SNAPSHOT":
+            connection.commit()

    def get_isolation_level(self, connection):
        if self.server_version_info < MS_2005_VERSION:
@@ -2515,13 +2527,12 @@ class MSDialect(default.DefaultDialect):
                )
                coltype = sqltypes.NULLTYPE
            else:
-                if (
-                    issubclass(coltype, sqltypes.Numeric)
-                    and coltype is not MSReal
-                ):
-                    kwargs["scale"] = numericscale
+                if issubclass(coltype, sqltypes.Numeric):
                    kwargs["precision"] = numericprec

+                    if not issubclass(coltype, sqltypes.Float):
+                        kwargs["scale"] = numericscale
+
                coltype = coltype(**kwargs)
            cdict = {
                "name": name,
@@ -56,10 +56,10 @@ Pass through exact Pyodbc string
 A PyODBC connection string can also be sent exactly as specified in
 `ConnectionStrings <https://code.google.com/p/pyodbc/wiki/ConnectionStrings>`_
 into the driver using the parameter ``odbc_connect``.  The delimeters must be
-URL escaped, however, as illustrated below using ``urllib.quote_plus``::
+URL escaped, however, as illustrated below using ``urllib.parse.quote_plus``::

    import urllib
-    params = urllib.quote_plus("DRIVER={SQL Server Native Client 10.0};SERVER=dagger;DATABASE=test;UID=user;PWD=password")
+    params = urllib.parse.quote_plus("DRIVER={SQL Server Native Client 10.0};SERVER=dagger;DATABASE=test;UID=user;PWD=password")

    engine = create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)

@@ -2543,8 +2543,12 @@ class MySQLDialect(default.DefaultDialect):
        # preserves the original table/schema casing, but SHOW CREATE
        # TABLE does not.   this problem is not in lower_case_table_names=1,
        # but use case-insensitive matching for these two modes in any case.
+
        if self._casing in (1, 2):
-            lower = str.lower
+
+            def lower(s):
+                return s.lower()
+
        else:
            # if on case sensitive, there can be two tables referenced
            # with the same name different casing, so we need to use
@@ -10,40 +10,9 @@ r"""
    :name: MySQL Connector/Python
    :dbapi: myconnpy
    :connectstring: mysql+mysqlconnector://<user>:<password>@<host>[:<port>]/<dbname>
-    :url: http://dev.mysql.com/downloads/connector/python/
+    :url: https://pypi.org/project/mysql-connector-python/


-Current Issues
--------------
-
-The mysqlconnector driver has many issues that have gone unresolved
-for many years and it recommended that mysqlclient or pymysql be used
-if possible; as of June 27, 2018:
-
-* the values in cursor.description are randomly sent as either bytes
-  or text with no discernible pattern, so the dialect must test these
-  individually and attempt to decode
-
-* has been observed to leak interpreter memory (likely at the C code level)
-  under scenarios that do not leak memory when using mysqlclient
-
-* Under Python 2, the driver does not support SQL statements that contain
-  non-ascii characters within the SQL text, making it impossible to support
-  schema objects with non-ascii names; an ascii encoding error is raised.
-
-* additional random bytes-returned issues occur when running under MySQL 8.0
-  only
-
-* The driver does not accept the "utf8mb4" or "utf8mb3" charset parameters,
-  only "utf8", even though MySQL itself has deprecated this symbol
-
-* The driver produces deadlocks when trying to make use of SELECT..FOR UPDATE,
-  the reason is unknown.
-
-This list should be updated as these issues are resolved either in the
-upstream mysql-connector-python driver or if appropriate usage patterns
-are contributed to SQLAlchemy.
-
 """  # noqa

 import re
@@ -19,6 +19,7 @@ from .base import DOUBLE_PRECISION
 from .base import FLOAT
 from .base import INTERVAL
 from .base import LONG
+from .base import NCHAR
 from .base import NCLOB
 from .base import NUMBER
 from .base import NVARCHAR
@@ -36,6 +37,7 @@ __all__ = (
    "VARCHAR",
    "NVARCHAR",
    "CHAR",
+    "NCHAR",
    "DATE",
    "NUMBER",
    "BLOB",
@@ -364,6 +364,7 @@ from ...types import CHAR
 from ...types import CLOB
 from ...types import FLOAT
 from ...types import INTEGER
+from ...types import NCHAR
 from ...types import NVARCHAR
 from ...types import TIMESTAMP
 from ...types import VARCHAR
@@ -523,6 +524,7 @@ ischema_names = {
    "VARCHAR2": VARCHAR,
    "NVARCHAR2": NVARCHAR,
    "CHAR": CHAR,
+    "NCHAR": NCHAR,
    "DATE": DATE,
    "NUMBER": NUMBER,
    "BLOB": BLOB,
@@ -1514,7 +1516,7 @@ class OracleDialect(default.DefaultDialect):
            elif coltype == "FLOAT":
                # TODO: support "precision" here as "binary_precision"
                coltype = FLOAT()
-            elif coltype in ("VARCHAR2", "NVARCHAR2", "CHAR"):
+            elif coltype in ("VARCHAR2", "NVARCHAR2", "CHAR", "NCHAR"):
                coltype = self.ischema_names.get(coltype)(length)
            elif "WITH TIME ZONE" in coltype:
                coltype = TIMESTAMP(timezone=True)
@@ -929,6 +929,7 @@ from ...sql import compiler
 from ...sql import elements
 from ...sql import expression
 from ...sql import sqltypes
+from ...sql import util as sql_util
 from ...types import BIGINT
 from ...types import BOOLEAN
 from ...types import CHAR
@@ -1681,10 +1682,11 @@ class PGCompiler(compiler.SQLCompiler):
            tmp = " FOR UPDATE"

        if select._for_update_arg.of:
-            tables = util.OrderedSet(
-                c.table if isinstance(c, expression.ColumnClause) else c
-                for c in select._for_update_arg.of
-            )
+
+            tables = util.OrderedSet()
+            for c in select._for_update_arg.of:
+                tables.update(sql_util.surface_selectables_only(c))
+
            tmp += " OF " + ", ".join(
                self.process(table, ashint=True, use_schema=False, **kw)
                for table in tables
@@ -147,44 +147,12 @@ def create_engine(*args, **kwargs):
        columns to accommodate Python Unicode objects directly as though the
        datatype were the :class:`.Unicode` type.

-          .. deprecated:: The :paramref:`.create_engine.convert_unicode` flag
-             and related Unicode conversion features are legacy Python 2
-             mechanisms which no longer have relevance under Python 3.
-             As all modern DBAPIs now support Python Unicode fully even
-             under Python 2, these flags will be removed in an upcoming
-             release.
-
-        .. note::
-
-            SQLAlchemy's unicode-conversion flags and features only apply
-            to Python 2; in Python 3, all string objects are Unicode objects.
-            For this reason, as well as the fact that virtually all modern
-            DBAPIs now support Unicode natively even under Python 2,
-            the :paramref:`.Engine.convert_unicode` flag is inherently a
-            legacy feature.
-
-        .. note::
-
-            This flag does **not** imply that SQLAlchemy's unicode-conversion
-            services will be used, as all modern DBAPIs already handle
-            unicode natively; in most cases it only indicates that the
-            :class:`.String` datatype will return Python unicode objects,
-            rather than plain strings.   The :class:`.String` datatype itself
-            has additional options to force the usage of SQLAlchemy's unicode
-            converters.
-
-        .. note::
-
-            This flag does **not** impact "raw" SQL statements that have no
-            typing information set up; that is, if the :class:`.String`
-            datatype is not used, no unicode behavior is implied.
-
-        .. seealso::
-
-            :paramref:`.String.convert_unicode` - the flag local to the
-            :class:`.String` datatype has additional options
-            which can force unicode handling on a per-type basis.
+        .. deprecated:: 1.3

+            The :paramref:`.create_engine.convert_unicode` parameter
+            is deprecated and will be removed in a future release.
+            All modern DBAPIs now support Python Unicode directly and this
+            parameter is unnecessary.

    :param creator: a callable which returns a DBAPI connection.
        This creation function will be passed to the underlying
@@ -418,11 +418,17 @@ class DefaultDialect(interfaces.Dialect):
        return sqltypes.adapt_type(typeobj, self.colspecs)

    def reflecttable(
-        self, connection, table, include_columns, exclude_columns, **opts
+        self,
+        connection,
+        table,
+        include_columns,
+        exclude_columns,
+        resolve_fks,
+        **opts
    ):
        insp = reflection.Inspector.from_engine(connection)
        return insp.reflecttable(
-            table, include_columns, exclude_columns, **opts
+            table, include_columns, exclude_columns, resolve_fks, **opts
        )

    def get_pk_constraint(self, conn, table_name, schema=None, **kw):
@@ -197,7 +197,7 @@ class Dialect(object):
        pass

    def reflecttable(
-        self, connection, table, include_columns, exclude_columns
+        self, connection, table, include_columns, exclude_columns, resolve_fks
    ):
        """Load table description from the database.

@@ -561,7 +561,12 @@ class Inspector(object):
        )

    def reflecttable(
-        self, table, include_columns, exclude_columns=(), _extend_on=None
+        self,
+        table,
+        include_columns,
+        exclude_columns=(),
+        resolve_fks=True,
+        _extend_on=None,
    ):
        """Given a Table object, load its internal constructs based on
        introspection.
@@ -650,6 +655,7 @@ class Inspector(object):
            table,
            cols_by_orig_name,
            exclude_columns,
+            resolve_fks,
            _extend_on,
            reflection_options,
        )
@@ -783,6 +789,7 @@ class Inspector(object):
        table,
        cols_by_orig_name,
        exclude_columns,
+        resolve_fks,
        _extend_on,
        reflection_options,
    ):
@@ -806,29 +813,31 @@ class Inspector(object):
            referred_columns = fkey_d["referred_columns"]
            refspec = []
            if referred_schema is not None:
-                sa_schema.Table(
-                    referred_table,
-                    table.metadata,
-                    autoload=True,
-                    schema=referred_schema,
-                    autoload_with=self.bind,
-                    _extend_on=_extend_on,
-                    **reflection_options
-                )
+                if resolve_fks:
+                    sa_schema.Table(
+                        referred_table,
+                        table.metadata,
+                        autoload=True,
+                        schema=referred_schema,
+                        autoload_with=self.bind,
+                        _extend_on=_extend_on,
+                        **reflection_options
+                    )
                for column in referred_columns:
                    refspec.append(
                        ".".join([referred_schema, referred_table, column])
                    )
            else:
-                sa_schema.Table(
-                    referred_table,
-                    table.metadata,
-                    autoload=True,
-                    autoload_with=self.bind,
-                    schema=sa_schema.BLANK_SCHEMA,
-                    _extend_on=_extend_on,
-                    **reflection_options
-                )
+                if resolve_fks:
+                    sa_schema.Table(
+                        referred_table,
+                        table.metadata,
+                        autoload=True,
+                        autoload_with=self.bind,
+                        schema=sa_schema.BLANK_SCHEMA,
+                        _extend_on=_extend_on,
+                        **reflection_options
+                    )
                for column in referred_columns:
                    refspec.append(".".join([referred_table, column]))
            if "options" in fkey_d:
@@ -120,8 +120,12 @@ class URL(object):
            and self.host == other.host
            and self.database == other.database
            and self.query == other.query
+            and self.port == other.port
        )

+    def __ne__(self, other):
+        return not self == other
+
    @property
    def password(self):
        if self.password_original is None:
@@ -76,7 +76,7 @@ class SQLAlchemyError(Exception):
        return self._sql_message(compat.py3k)

    def __unicode__(self):
-        return self._sql_message(True)
+        return self._sql_message(as_unicode=True)


 class ArgumentError(SQLAlchemyError):
@@ -346,14 +346,20 @@ class StatementError(SQLAlchemyError):

        details = [self._message(as_unicode=as_unicode)]
        if self.statement:
-            details.append("[SQL: %r]" % self.statement)
+            if not as_unicode and not compat.py3k:
+                stmt_detail = "[SQL: %s]" % compat.safe_bytestring(
+                    self.statement
+                )
+            else:
+                stmt_detail = "[SQL: %s]" % self.statement
+            details.append(stmt_detail)
            if self.params:
                params_repr = util._repr_params(self.params, 10)
                details.append("[parameters: %r]" % params_repr)
        code_str = self._code_str()
        if code_str:
            details.append(code_str)
-        return " ".join(["(%s)" % det for det in self.detail] + details)
+        return "\n".join(["(%s)" % det for det in self.detail] + details)


 class DBAPIError(StatementError):
@@ -140,6 +140,10 @@ class AssociationProxy(interfaces.InspectionAttrInfo):

         .. versionadded:: 1.3

+         .. seealso::
+
+            :ref:`cascade_scalar_deletes` - complete usage example
+
        :param getset_factory: Optional.  Proxied attribute access is
          automatically handled by routines that get and set values based on
          the `attr` argument for this proxy.
@@ -509,7 +509,7 @@ reference the instrumented attribute back to the hybrid object::

        last_name = Column(String)

-        @FirstNameOnly.overrides.expression
+        @FirstNameOnly.name.overrides.expression
        def name(cls):
            return func.concat(cls.first_name, ' ', cls.last_name)

@@ -1724,6 +1724,17 @@ def get_history(obj, key, passive=PASSIVE_OFF):
    """Return a :class:`.History` record for the given object
    and attribute key.

+    This is the **pre-flush** history for a given attribute, which is
+    reset each time the :class:`.Session` flushes changes to the
+    current database transaction.
+
+    .. note::
+
+        Prefer to use the :attr:`.AttributeState.history` and
+        :meth:`.AttributeState.load_history` accessors to retrieve the
+        :class:`.History` for instance attributes.
+
+
    :param obj: an object whose class is instrumented by the
      attributes package.

@@ -1735,6 +1746,13 @@ def get_history(obj, key, passive=PASSIVE_OFF):
       :attr:`.PASSIVE_OFF` indicating all necessary SQL
       should be emitted.

+    .. seealso::
+
+        :attr:`.AttributeState.history`
+
+        :meth:`.AttributeState.load_history` - retrieve history
+        using loader callables if the value is not locally present.
+
    """
    if passive is True:
        util.warn_deprecated(
@@ -39,6 +39,10 @@ class DescriptorProperty(MapperProperty):
            expire_missing = True
            collection = False

+            @property
+            def uses_objects(self):
+                return getattr(mapper.class_, prop.name).impl.uses_objects
+
            def __init__(self, key):
                self.key = key

@@ -507,7 +507,7 @@ def _collect_insert_commands(
                and hasattr(value, "__clause_element__")
                or isinstance(value, sql.ClauseElement)
            ):
-                value_params[col.key] = (
+                value_params[col] = (
                    value.__clause_element__()
                    if hasattr(value, "__clause_element__")
                    else value
@@ -525,7 +525,7 @@ def _collect_insert_commands(
            for colkey in (
                mapper._insert_cols_as_none[table]
                .difference(params)
-                .difference(value_params)
+                .difference([c.key for c in value_params])
            ):
                params[colkey] = None

@@ -932,6 +932,7 @@ def _emit_update_statements(
                        c,
                        c.context.compiled_parameters[0],
                        value_params,
+                        True,
                    )
                rows += c.rowcount
                check_rowcount = assert_singlerow
@@ -963,6 +964,7 @@ def _emit_update_statements(
                            c,
                            c.context.compiled_parameters[0],
                            value_params,
+                            True,
                        )
                    rows += c.rowcount
            else:
@@ -998,6 +1000,7 @@ def _emit_update_statements(
                            c,
                            c.context.compiled_parameters[0],
                            value_params,
+                            True,
                        )

        if check_rowcount:
@@ -1086,6 +1089,7 @@ def _emit_insert_statements(
                            c,
                            last_inserted_params,
                            value_params,
+                            False,
                        )
                    else:
                        _postfetch_bulk_save(mapper_rec, state_dict, table)
@@ -1117,14 +1121,16 @@ def _emit_insert_statements(
                    )

                primary_key = result.context.inserted_primary_key
-
                if primary_key is not None:
                    # set primary key attributes
                    for pk, col in zip(
                        primary_key, mapper._pks_by_table[table]
                    ):
                        prop = mapper_rec._columntoproperty[col]
-                        if state_dict.get(prop.key) is None:
+                        if pk is not None and (
+                            col in value_params
+                            or state_dict.get(prop.key) is None
+                        ):
                            state_dict[prop.key] = pk
                if bookkeeping:
                    if state:
@@ -1137,6 +1143,7 @@ def _emit_insert_statements(
                            result,
                            result.context.compiled_parameters[0],
                            value_params,
+                            False,
                        )
                    else:
                        _postfetch_bulk_save(mapper_rec, state_dict, table)
@@ -1461,7 +1468,15 @@ def _postfetch_post_update(


 def _postfetch(
-    mapper, uowtransaction, table, state, dict_, result, params, value_params
+    mapper,
+    uowtransaction,
+    table,
+    state,
+    dict_,
+    result,
+    params,
+    value_params,
+    isupdate,
 ):
    """Expire attributes in need of newly persisted database state,
    after an INSERT or UPDATE statement has proceeded for that
@@ -1511,6 +1526,18 @@ def _postfetch(
            state, uowtransaction, load_evt_attrs
        )

+    if isupdate and value_params:
+        # explicitly suit the use case specified by
+        # [ticket:3801], PK SQL expressions for UPDATE on non-RETURNING
+        # database which are set to themselves in order to do a version bump.
+        postfetch_cols.extend(
+            [
+                col
+                for col in value_params
+                if col.primary_key and col not in returning_cols
+            ]
+        )
+
    if postfetch_cols:
        state._expire_attributes(
            state.dict,
@@ -883,6 +883,9 @@ class Query(object):

            some_object = session.query(VersionedFoo).get((5, 10))

+            some_object = session.query(VersionedFoo).get(
+                {"id": 5, "version_id": 10})
+
        :meth:`~.Query.get` is special in that it provides direct
        access to the identity map of the owning :class:`.Session`.
        If the given primary key identifier is present
@@ -918,14 +921,37 @@ class Query(object):
        before querying the database.  See :doc:`/orm/loading_relationships`
        for further details on relationship loading.

-        :param ident: A scalar or tuple value representing
-         the primary key.   For a composite primary key,
-         the order of identifiers corresponds in most cases
-         to that of the mapped :class:`.Table` object's
-         primary key columns.  For a :func:`.mapper` that
-         was given the ``primary key`` argument during
-         construction, the order of identifiers corresponds
-         to the elements present in this collection.
+        :param ident: A scalar, tuple, or dictionary representing the
+         primary key.  For a composite (e.g. multiple column) primary key,
+         a tuple or dictionary should be passed.
+
+         For a single-column primary key, the scalar calling form is typically
+         the most expedient.  If the primary key of a row is the value "5",
+         the call looks like::
+
+            my_object = query.get(5)
+
+         The tuple form contains primary key values typically in
+         the order in which they correspond to the mapped :class:`.Table`
+         object's primary key columns, or if the
+         :paramref:`.Mapper.primary_key` configuration parameter were used, in
+         the order used for that parameter. For example, if the primary key
+         of a row is represented by the integer
+         digits "5, 10" the call would look like::
+
+             my_object = query.get((5, 10))
+
+         The dictionary form should include as keys the mapped attribute names
+         corresponding to each element of the primary key.  If the mapped class
+         has the attributes ``id``, ``version_id`` as the attributes which
+         store the object's primary key value, the call would look like::
+
+            my_object = query.get({"id": 5, "version_id": 10})
+
+         .. versionadded:: 1.3 the :meth:`.Query.get` method now optionally
+            accepts a dictionary of attribute names to values in order to
+            indicate a primary key identifier.
+

        :return: The object instance, or ``None``.

@@ -991,10 +1017,12 @@ class Query(object):
        if hasattr(primary_key_identity, "__composite_values__"):
            primary_key_identity = primary_key_identity.__composite_values__()

-        primary_key_identity = util.to_list(primary_key_identity)
-
        mapper = self._only_full_mapper_zero("get")

+        is_dict = isinstance(primary_key_identity, dict)
+        if not is_dict:
+            primary_key_identity = util.to_list(primary_key_identity)
+
        if len(primary_key_identity) != len(mapper.primary_key):
            raise sa_exc.InvalidRequestError(
                "Incorrect number of values in identifier to formulate "
@@ -1002,6 +1030,24 @@ class Query(object):
                % ",".join("'%s'" % c for c in mapper.primary_key)
            )

+        if is_dict:
+            try:
+                primary_key_identity = list(
+                    primary_key_identity[prop.key]
+                    for prop in mapper._identity_key_props
+                )
+
+            except KeyError:
+                raise sa_exc.InvalidRequestError(
+                    "Incorrect names of values in identifier to formulate "
+                    "primary key for query.get(); primary key attribute names"
+                    " are %s"
+                    % ",".join(
+                        "'%s'" % prop.key
+                        for prop in mapper._identity_key_props
+                    )
+                )
+
        if (
            not self._populate_existing
            and not mapper.always_refresh
@@ -182,7 +182,14 @@ class RelationshipProperty(StrategizedProperty):
        callables that evaluate the string as Python code, using the
        Declarative class-registry as a namespace.  This allows the lookup of
        related classes to be automatic via their string name, and removes the
-        need to import related classes at all into the local module space::
+        need for related classes to be imported into the local module space
+        before the dependent classes have been declared.  It is still required
+        that the modules in which these related classes appear are imported
+        anywhere in the application at some point before the related mappings
+        are actually used, else a lookup error will be raised when the
+        :func:`.relationship` attempts to resolve the string reference to the
+        related class.    An example of a string- resolved class is as
+        follows::

            from sqlalchemy.ext.declarative import declarative_base

@@ -1827,7 +1827,14 @@ class Session(_SessionClassMethods):
            states, self, to_transient=to_transient
        )

-    def _register_newly_persistent(self, states):
+    def _register_persistent(self, states):
+        """Register all persistent objects from a flush.
+
+        This is used both for pending objects moving to the persistent
+        state as well as already persistent objects.
+
+        """
+
        pending_to_persistent = self.dispatch.pending_to_persistent or None
        for state in states:
            mapper = _state_mapper(state)
@@ -1871,6 +1878,9 @@ class Session(_SessionClassMethods):
                    )
                    state.key = instance_key

+                # there can be an existing state in the identity map
+                # that is replaced when the primary keys of two instances
+                # are swapped; see test/orm/test_naturalpks.py -> test_reverse
                self.identity_map.replace(state)
                state._orphaned_outside_of_session = False

@@ -1881,7 +1891,7 @@ class Session(_SessionClassMethods):
        self._register_altered(states)

        if pending_to_persistent is not None:
-            for state in states:
+            for state in states.intersection(self._new):
                pending_to_persistent(self, state.obj())

        # remove from new last, might be the last strong ref
@@ -880,12 +880,21 @@ class AttributeState(object):

    @property
    def history(self):
-        """Return the current pre-flush change history for
+        """Return the current **pre-flush** change history for
        this attribute, via the :class:`.History` interface.

        This method will **not** emit loader callables if the value of the
        attribute is unloaded.

+        .. note::
+
+            The attribute history system tracks changes on a **per flush
+            basis**. Each time the :class:`.Session` is flushed, the history
+            of each attribute is reset to empty.   The :class:`.Session` by
+            default autoflushes each time a :class:`.Query` is invoked.  For
+            options on how to control this, see :ref:`session_flushing`.
+
+
        .. seealso::

            :meth:`.AttributeState.load_history` - retrieve history
@@ -897,12 +906,20 @@ class AttributeState(object):
        return self.state.get_history(self.key, PASSIVE_NO_INITIALIZE)

    def load_history(self):
-        """Return the current pre-flush change history for
+        """Return the current **pre-flush** change history for
        this attribute, via the :class:`.History` interface.

        This method **will** emit loader callables if the value of the
        attribute is unloaded.

+        .. note::
+
+            The attribute history system tracks changes on a **per flush
+            basis**. Each time the :class:`.Session` is flushed, the history
+            of each attribute is reset to empty.   The :class:`.Session` by
+            default autoflushes each time a :class:`.Query` is invoked.  For
+            options on how to control this, see :ref:`session_flushing`.
+
        .. seealso::

            :attr:`.AttributeState.history`
@@ -595,6 +595,10 @@ class LazyLoader(AbstractRelationshipLoader, util.MemoizedSlots):

        def visit_bindparam(bindparam):
            bindparam.unique = False
+
+        visitors.traverse(criterion, {}, {"bindparam": visit_bindparam})
+
+        def visit_bindparam(bindparam):
            if bindparam._identifying_key in bind_to_col:
                params.append(
                    (
@@ -440,7 +440,7 @@ class UOWTransaction(object):
        if isdel:
            self.session._remove_newly_deleted(isdel)
        if other:
-            self.session._register_newly_persistent(other)
+            self.session._register_persistent(other)


 class IterateMappersMixin(object):
@@ -21,6 +21,7 @@ from .expression import column  # noqa
 from .expression import ColumnCollection  # noqa
 from .expression import ColumnElement  # noqa
 from .expression import CompoundSelect  # noqa
+from .expression import cte  # noqa
 from .expression import Delete  # noqa
 from .expression import delete  # noqa
 from .expression import desc  # noqa
@@ -409,7 +409,12 @@ def _append_param_parameter(
            compiler.returning.append(c)
            value = compiler.process(value.self_group(), **kw)
        else:
-            compiler.postfetch.append(c)
+            # postfetch specifically means, "we can SELECT the row we just
+            # inserted by primary key to get back the server generated
+            # defaults". so by definition this can't be used to get the primary
+            # key value back, because we need to have it ahead of time.
+            if not c.primary_key:
+                compiler.postfetch.append(c)
            value = compiler.process(value.self_group(), **kw)
    values.append((c, value))

@@ -39,6 +39,7 @@ __all__ = [
    "case",
    "cast",
    "column",
+    "cte",
    "delete",
    "desc",
    "distinct",
@@ -148,7 +149,6 @@ from .functions import FunctionElement  # noqa
 from .functions import modifier  # noqa
 from .selectable import _interpret_as_from  # noqa
 from .selectable import Alias  # noqa
-from .selectable import alias  # noqa
 from .selectable import CompoundSelect  # noqa
 from .selectable import CTE  # noqa
 from .selectable import Exists  # noqa
@@ -160,7 +160,6 @@ from .selectable import HasPrefixes  # noqa
 from .selectable import HasSuffixes  # noqa
 from .selectable import Join  # noqa
 from .selectable import Lateral  # noqa
-from .selectable import lateral  # noqa
 from .selectable import ScalarSelect  # noqa
 from .selectable import Select  # noqa
 from .selectable import Selectable  # noqa
@@ -168,7 +167,6 @@ from .selectable import SelectBase  # noqa
 from .selectable import subquery  # noqa
 from .selectable import TableClause  # noqa
 from .selectable import TableSample  # noqa
-from .selectable import tablesample  # noqa
 from .selectable import TextAsFrom  # noqa
 from .visitors import Visitable  # noqa
 from ..util.langhelpers import public_factory  # noqa
@@ -182,6 +180,9 @@ from ..util.langhelpers import public_factory  # noqa
 all_ = public_factory(CollectionAggregate._create_all, ".expression.all_")
 any_ = public_factory(CollectionAggregate._create_any, ".expression.any_")
 and_ = public_factory(BooleanClauseList.and_, ".expression.and_")
+alias = public_factory(Alias._factory, ".expression.alias")
+tablesample = public_factory(TableSample._factory, ".expression.tablesample")
+lateral = public_factory(Lateral._factory, ".expression.lateral")
 or_ = public_factory(BooleanClauseList.or_, ".expression.or_")
 bindparam = public_factory(BindParameter, ".expression.bindparam")
 select = public_factory(Select, ".expression.select")
@@ -193,6 +194,7 @@ within_group = public_factory(WithinGroup, ".expression.within_group")
 label = public_factory(Label, ".expression.label")
 case = public_factory(Case, ".expression.case")
 cast = public_factory(Cast, ".expression.cast")
+cte = public_factory(CTE._factory, ".expression.cte")
 extract = public_factory(Extract, ".exp  # noqaression.extract")
 tuple_ = public_factory(Tuple, ".expression.tuple_")
 except_ = public_factory(CompoundSelect._create_except, ".expression.except_")
@@ -301,7 +301,7 @@ class FunctionElement(Executable, ColumnElement, FromClause):

        """

-        return Alias(self, name)
+        return Alias._construct(self, name)

    def select(self):
        """Produce a :func:`~.expression.select` construct
@@ -311,6 +311,24 @@ class Table(DialectKWArgs, SchemaItem, TableClause):
        ``Table`` object. Defaults to ``None`` which indicates all columns
        should be reflected.

+    :param resolve_fks: Whether or not to reflect :class:`.Table` objects
+        related to this one via :class:`.ForeignKey` objects, when
+        :paramref:`.Table.autoload` or :paramref:`.Table.autoload_with` is
+        specified.   Defaults to True.  Set to False to disable reflection of
+        related tables as :class:`.ForeignKey` objects are encountered; may be
+        used either to save on SQL calls or to avoid issues with related tables
+        that can't be accessed. Note that if a related table is already present
+        in the :class:`.MetaData` collection, or becomes present later, a
+        :class:`.ForeignKey` object associated with this :class:`.Table` will
+        resolve to that table normally.
+
+        .. versionadded:: 1.3
+
+        .. seealso::
+
+            :paramref:`.MetaData.reflect.resolve_fks`
+
+
    :param info: Optional data dictionary which will be populated into the
        :attr:`.SchemaItem.info` attribute of this object.

@@ -537,6 +555,7 @@ class Table(DialectKWArgs, SchemaItem, TableClause):
        kwargs.pop("autoload_replace", True)
        _extend_on = kwargs.pop("_extend_on", None)

+        resolve_fks = kwargs.pop("resolve_fks", True)
        include_columns = kwargs.pop("include_columns", None)

        self.implicit_returning = kwargs.pop("implicit_returning", True)
@@ -559,7 +578,11 @@ class Table(DialectKWArgs, SchemaItem, TableClause):
        # circular foreign keys
        if autoload:
            self._autoload(
-                metadata, autoload_with, include_columns, _extend_on=_extend_on
+                metadata,
+                autoload_with,
+                include_columns,
+                _extend_on=_extend_on,
+                resolve_fks=resolve_fks,
            )

        # initialize all the column, etc. objects.  done after reflection to
@@ -572,6 +595,7 @@ class Table(DialectKWArgs, SchemaItem, TableClause):
        autoload_with,
        include_columns,
        exclude_columns=(),
+        resolve_fks=True,
        _extend_on=None,
    ):

@@ -581,6 +605,7 @@ class Table(DialectKWArgs, SchemaItem, TableClause):
                self,
                include_columns,
                exclude_columns,
+                resolve_fks,
                _extend_on=_extend_on,
            )
        else:
@@ -597,6 +622,7 @@ class Table(DialectKWArgs, SchemaItem, TableClause):
                self,
                include_columns,
                exclude_columns,
+                resolve_fks,
                _extend_on=_extend_on,
            )

@@ -636,6 +662,8 @@ class Table(DialectKWArgs, SchemaItem, TableClause):

        include_columns = kwargs.pop("include_columns", None)

+        resolve_fks = kwargs.pop("resolve_fks", True)
+
        if include_columns is not None:
            for c in self.c:
                if c.name not in include_columns:
@@ -666,6 +694,7 @@ class Table(DialectKWArgs, SchemaItem, TableClause):
                autoload_with,
                include_columns,
                exclude_columns,
+                resolve_fks,
                _extend_on=_extend_on,
            )

@@ -2906,12 +2935,12 @@ class CheckConstraint(ColumnCollectionConstraint):
        r"""Construct a CHECK constraint.

        :param sqltext:
-          A string containing the constraint definition, which will be used
-          verbatim, or a SQL expression construct.   If given as a string,
-          the object is converted to a :class:`.Text` object.   If the textual
-          string includes a colon character, escape this using a backslash::
+         A string containing the constraint definition, which will be used
+         verbatim, or a SQL expression construct.   If given as a string,
+         the object is converted to a :func:`.text` object. If the textual
+         string includes a colon character, escape this using a backslash::

-            CheckConstraint(r"foo ~ E'a(?\:b|c)d")
+           CheckConstraint(r"foo ~ E'a(?\:b|c)d")

        :param name:
          Optional, the in-database name of the constraint.
@@ -4065,6 +4094,7 @@ class MetaData(SchemaItem):
        only=None,
        extend_existing=False,
        autoload_replace=True,
+        resolve_fks=True,
        **dialect_kwargs
    ):
        r"""Load all available table definitions from the database.
@@ -4111,6 +4141,26 @@ class MetaData(SchemaItem):

          .. versionadded:: 0.9.1

+        :param resolve_fks: if True, reflect :class:`.Table` objects linked
+         to :class:`.ForeignKey` objects located in each :class:`.Table`.
+         For :meth:`.MetaData.reflect`, this has the effect of reflecting
+         related tables that might otherwise not be in the list of tables
+         being reflected, for example if the referenced table is in a
+         different schema or is omitted via the
+         :paramref:`.MetaData.reflect.only` parameter.  When False,
+         :class:`.ForeignKey` objects are not followed to the :class:`.Table`
+         in which they link, however if the related table is also part of the
+         list of tables that would be reflected in any case, the
+         :class:`.ForeignKey` object will still resolve to its related
+         :class:`.Table` after the :meth:`.MetaData.reflect` operation is
+         complete.   Defaults to True.
+
+         .. versionadded:: 1.3.0
+
+         .. seealso::
+
+            :paramref:`.Table.resolve_fks`
+
        :param \**dialect_kwargs: Additional keyword arguments not mentioned
         above are dialect specific, and passed in the form
         ``<dialectname>_<argname>``.  See the documentation regarding an
@@ -4133,6 +4183,7 @@ class MetaData(SchemaItem):
                "autoload_with": conn,
                "extend_existing": extend_existing,
                "autoload_replace": autoload_replace,
+                "resolve_fks": resolve_fks,
                "_extend_on": set(),
            }

@@ -133,117 +133,6 @@ def subquery(alias, *args, **kwargs):
    return Select(*args, **kwargs).alias(alias)


-def alias(selectable, name=None, flat=False):
-    """Return an :class:`.Alias` object.
-
-    An :class:`.Alias` represents any :class:`.FromClause`
-    with an alternate name assigned within SQL, typically using the ``AS``
-    clause when generated, e.g. ``SELECT * FROM table AS aliasname``.
-
-    Similar functionality is available via the
-    :meth:`~.FromClause.alias` method
-    available on all :class:`.FromClause` subclasses.
-
-    When an :class:`.Alias` is created from a :class:`.Table` object,
-    this has the effect of the table being rendered
-    as ``tablename AS aliasname`` in a SELECT statement.
-
-    For :func:`.select` objects, the effect is that of creating a named
-    subquery, i.e. ``(select ...) AS aliasname``.
-
-    The ``name`` parameter is optional, and provides the name
-    to use in the rendered SQL.  If blank, an "anonymous" name
-    will be deterministically generated at compile time.
-    Deterministic means the name is guaranteed to be unique against
-    other constructs used in the same statement, and will also be the
-    same name for each successive compilation of the same statement
-    object.
-
-    :param selectable: any :class:`.FromClause` subclass,
-        such as a table, select statement, etc.
-
-    :param name: string name to be assigned as the alias.
-        If ``None``, a name will be deterministically generated
-        at compile time.
-
-    :param flat: Will be passed through to if the given selectable
-     is an instance of :class:`.Join` - see :meth:`.Join.alias`
-     for details.
-
-     .. versionadded:: 0.9.0
-
-    """
-    return _interpret_as_from(selectable).alias(name=name, flat=flat)
-
-
-def lateral(selectable, name=None):
-    """Return a :class:`.Lateral` object.
-
-    :class:`.Lateral` is an :class:`.Alias` subclass that represents
-    a subquery with the LATERAL keyword applied to it.
-
-    The special behavior of a LATERAL subquery is that it appears in the
-    FROM clause of an enclosing SELECT, but may correlate to other
-    FROM clauses of that SELECT.   It is a special case of subquery
-    only supported by a small number of backends, currently more recent
-    PostgreSQL versions.
-
-    .. versionadded:: 1.1
-
-    .. seealso::
-
-        :ref:`lateral_selects` -  overview of usage.
-
-    """
-    return _interpret_as_from(selectable).lateral(name=name)
-
-
-def tablesample(selectable, sampling, name=None, seed=None):
-    """Return a :class:`.TableSample` object.
-
-    :class:`.TableSample` is an :class:`.Alias` subclass that represents
-    a table with the TABLESAMPLE clause applied to it.
-    :func:`~.expression.tablesample`
-    is also available from the :class:`.FromClause` class via the
-    :meth:`.FromClause.tablesample` method.
-
-    The TABLESAMPLE clause allows selecting a randomly selected approximate
-    percentage of rows from a table. It supports multiple sampling methods,
-    most commonly BERNOULLI and SYSTEM.
-
-    e.g.::
-
-        from sqlalchemy import func
-
-        selectable = people.tablesample(
-                    func.bernoulli(1),
-                    name='alias',
-                    seed=func.random())
-        stmt = select([selectable.c.people_id])
-
-    Assuming ``people`` with a column ``people_id``, the above
-    statement would render as::
-
-        SELECT alias.people_id FROM
-        people AS alias TABLESAMPLE bernoulli(:bernoulli_1)
-        REPEATABLE (random())
-
-    .. versionadded:: 1.1
-
-    :param sampling: a ``float`` percentage between 0 and 100 or
-        :class:`.functions.Function`.
-
-    :param name: optional alias name
-
-    :param seed: any real-valued SQL expression.  When specified, the
-     REPEATABLE sub-clause is also rendered.
-
-    """
-    return _interpret_as_from(selectable).tablesample(
-        sampling, name=name, seed=seed
-    )
-
-
 class Selectable(ClauseElement):
    """mark a class as being selectable"""

@@ -535,7 +424,7 @@ class FromClause(Selectable):

        """

-        return Alias(self, name)
+        return Alias._construct(self, name)

    def lateral(self, name=None):
        """Return a LATERAL alias of this :class:`.FromClause`.
@@ -550,7 +439,7 @@ class FromClause(Selectable):
            :ref:`lateral_selects` -  overview of usage.

        """
-        return Lateral(self, name)
+        return Lateral._construct(self, name)

    def tablesample(self, sampling, name=None, seed=None):
        """Return a TABLESAMPLE alias of this :class:`.FromClause`.
@@ -565,7 +454,7 @@ class FromClause(Selectable):
            :func:`~.expression.tablesample` - usage guidelines and parameters

        """
-        return TableSample(self, sampling, name, seed)
+        return TableSample._construct(self, sampling, name, seed)

    def is_derived_from(self, fromclause):
        """Return True if this FromClause is 'derived' from the given
@@ -1329,7 +1218,69 @@ class Alias(FromClause):

    _is_from_container = True

-    def __init__(self, selectable, name=None):
+    def __init__(self, *arg, **kw):
+        raise NotImplementedError(
+            "The %s class is not intended to be constructed "
+            "directly.  Please use the %s() standalone "
+            "function or the %s() method available from appropriate "
+            "selectable objects."
+            % (
+                self.__class__.__name__,
+                self.__class__.__name__.lower(),
+                self.__class__.__name__.lower(),
+            )
+        )
+
+    @classmethod
+    def _construct(cls, *arg, **kw):
+        obj = cls.__new__(cls)
+        obj._init(*arg, **kw)
+        return obj
+
+    @classmethod
+    def _factory(cls, selectable, name=None, flat=False):
+        """Return an :class:`.Alias` object.
+
+        An :class:`.Alias` represents any :class:`.FromClause`
+        with an alternate name assigned within SQL, typically using the ``AS``
+        clause when generated, e.g. ``SELECT * FROM table AS aliasname``.
+
+        Similar functionality is available via the
+        :meth:`~.FromClause.alias` method
+        available on all :class:`.FromClause` subclasses.
+
+        When an :class:`.Alias` is created from a :class:`.Table` object,
+        this has the effect of the table being rendered
+        as ``tablename AS aliasname`` in a SELECT statement.
+
+        For :func:`.select` objects, the effect is that of creating a named
+        subquery, i.e. ``(select ...) AS aliasname``.
+
+        The ``name`` parameter is optional, and provides the name
+        to use in the rendered SQL.  If blank, an "anonymous" name
+        will be deterministically generated at compile time.
+        Deterministic means the name is guaranteed to be unique against
+        other constructs used in the same statement, and will also be the
+        same name for each successive compilation of the same statement
+        object.
+
+        :param selectable: any :class:`.FromClause` subclass,
+            such as a table, select statement, etc.
+
+        :param name: string name to be assigned as the alias.
+            If ``None``, a name will be deterministically generated
+            at compile time.
+
+        :param flat: Will be passed through to if the given selectable
+         is an instance of :class:`.Join` - see :meth:`.Join.alias`
+         for details.
+
+         .. versionadded:: 0.9.0
+
+        """
+        return _interpret_as_from(selectable).alias(name=name, flat=flat)
+
+    def _init(self, selectable, name=None):
        baseselectable = selectable
        while isinstance(baseselectable, Alias):
            baseselectable = baseselectable.element
@@ -1437,6 +1388,28 @@ class Lateral(Alias):
    __visit_name__ = "lateral"
    _is_lateral = True

+    @classmethod
+    def _factory(cls, selectable, name=None):
+        """Return a :class:`.Lateral` object.
+
+        :class:`.Lateral` is an :class:`.Alias` subclass that represents
+        a subquery with the LATERAL keyword applied to it.
+
+        The special behavior of a LATERAL subquery is that it appears in the
+        FROM clause of an enclosing SELECT, but may correlate to other
+        FROM clauses of that SELECT.   It is a special case of subquery
+        only supported by a small number of backends, currently more recent
+        PostgreSQL versions.
+
+        .. versionadded:: 1.1
+
+        .. seealso::
+
+            :ref:`lateral_selects` -  overview of usage.
+
+        """
+        return _interpret_as_from(selectable).lateral(name=name)
+

 class TableSample(Alias):
    """Represent a TABLESAMPLE clause.
@@ -1455,10 +1428,56 @@ class TableSample(Alias):

    __visit_name__ = "tablesample"

-    def __init__(self, selectable, sampling, name=None, seed=None):
+    @classmethod
+    def _factory(cls, selectable, sampling, name=None, seed=None):
+        """Return a :class:`.TableSample` object.
+
+        :class:`.TableSample` is an :class:`.Alias` subclass that represents
+        a table with the TABLESAMPLE clause applied to it.
+        :func:`~.expression.tablesample`
+        is also available from the :class:`.FromClause` class via the
+        :meth:`.FromClause.tablesample` method.
+
+        The TABLESAMPLE clause allows selecting a randomly selected approximate
+        percentage of rows from a table. It supports multiple sampling methods,
+        most commonly BERNOULLI and SYSTEM.
+
+        e.g.::
+
+            from sqlalchemy import func
+
+            selectable = people.tablesample(
+                        func.bernoulli(1),
+                        name='alias',
+                        seed=func.random())
+            stmt = select([selectable.c.people_id])
+
+        Assuming ``people`` with a column ``people_id``, the above
+        statement would render as::
+
+            SELECT alias.people_id FROM
+            people AS alias TABLESAMPLE bernoulli(:bernoulli_1)
+            REPEATABLE (random())
+
+        .. versionadded:: 1.1
+
+        :param sampling: a ``float`` percentage between 0 and 100 or
+            :class:`.functions.Function`.
+
+        :param name: optional alias name
+
+        :param seed: any real-valued SQL expression.  When specified, the
+         REPEATABLE sub-clause is also rendered.
+
+        """
+        return _interpret_as_from(selectable).tablesample(
+            sampling, name=name, seed=seed
+        )
+
+    def _init(self, selectable, sampling, name=None, seed=None):
        self.sampling = sampling
        self.seed = seed
-        super(TableSample, self).__init__(selectable, name=name)
+        super(TableSample, self)._init(selectable, name=name)

    @util.dependencies("sqlalchemy.sql.functions")
    def _get_method(self, functions):
@@ -1479,7 +1498,18 @@ class CTE(Generative, HasSuffixes, Alias):

    __visit_name__ = "cte"

-    def __init__(
+    @classmethod
+    def _factory(cls, selectable, name=None, recursive=False):
+        r"""Return a new :class:`.CTE`, or Common Table Expression instance.
+
+        Please see :meth:`.HasCte.cte` for detail on CTE usage.
+
+        """
+        return _interpret_as_from(selectable).cte(
+            name=name, recursive=recursive
+        )
+
+    def _init(
        self,
        selectable,
        name=None,
@@ -1493,7 +1523,7 @@ class CTE(Generative, HasSuffixes, Alias):
        self._restates = _restates
        if _suffixes:
            self._suffixes = _suffixes
-        super(CTE, self).__init__(selectable, name=name)
+        super(CTE, self)._init(selectable, name=name)

    def _copy_internals(self, clone=_clone, **kw):
        super(CTE, self)._copy_internals(clone, **kw)
@@ -1513,7 +1543,7 @@ class CTE(Generative, HasSuffixes, Alias):
                col._make_proxy(self)

    def alias(self, name=None, flat=False):
-        return CTE(
+        return CTE._construct(
            self.original,
            name=name,
            recursive=self.recursive,
@@ -1522,7 +1552,7 @@ class CTE(Generative, HasSuffixes, Alias):
        )

    def union(self, other):
-        return CTE(
+        return CTE._construct(
            self.original.union(other),
            name=self.name,
            recursive=self.recursive,
@@ -1531,7 +1561,7 @@ class CTE(Generative, HasSuffixes, Alias):
        )

    def union_all(self, other):
-        return CTE(
+        return CTE._construct(
            self.original.union_all(other),
            name=self.name,
            recursive=self.recursive,
@@ -1705,7 +1735,7 @@ class HasCTE(object):
            :meth:`.HasCTE.cte`.

        """
-        return CTE(self, name=name, recursive=recursive)
+        return CTE._construct(self, name=name, recursive=recursive)


 class FromGrouping(FromClause):
@@ -3197,6 +3227,14 @@ class Select(HasPrefixes, HasSuffixes, GenerativeSelect):
        self._correlate = set(clone(f) for f in self._correlate).union(
            self._correlate
        )
+
+        # do something similar for _correlate_except - this is a more
+        # unusual case but same idea applies
+        if self._correlate_except:
+            self._correlate_except = set(
+                clone(f) for f in self._correlate_except
+            ).union(self._correlate_except)
+
        # 4. clone other things.   The difficulty here is that Column
        # objects are not actually cloned, and refer to their original
        # .table, resulting in the wrong "from" parent after a clone
@@ -2214,8 +2214,13 @@ class JSON(Indexable, TypeEngine):

    @property
    def should_evaluate_none(self):
+        """Alias of :attr:`.JSON.none_as_null`"""
        return not self.none_as_null

+    @should_evaluate_none.setter
+    def should_evaluate_none(self, value):
+        self.none_as_null = not value
+
    @util.memoized_property
    def _str_impl(self):
        return String(_expect_unicode=True)
@@ -29,11 +29,13 @@ from .elements import ColumnElement
 from .elements import Null
 from .elements import UnaryExpression
 from .schema import Column
+from .selectable import Alias
 from .selectable import FromClause
 from .selectable import FromGrouping
 from .selectable import Join
 from .selectable import ScalarSelect
 from .selectable import SelectBase
+from .selectable import TableClause
 from .. import exc
 from .. import util

@@ -339,6 +341,20 @@ def surface_selectables(clause):
            stack.append(elem.element)


+def surface_selectables_only(clause):
+    stack = [clause]
+    while stack:
+        elem = stack.pop()
+        if isinstance(elem, (TableClause, Alias)):
+            yield elem
+        if isinstance(elem, Join):
+            stack.extend((elem.left, elem.right))
+        elif isinstance(elem, FromGrouping):
+            stack.append(elem.element)
+        elif isinstance(elem, ColumnClause):
+            stack.append(elem.table)
+
+
 def surface_column_elements(clause, include_scalar_selects=True):
    """traverse and yield only outer-exposed column elements, such as would
    be addressable in the WHERE clause of a SELECT if this element were
@@ -227,6 +227,15 @@ else:
            # error callback"
            return repr(text)[1:-1].decode()

+    def safe_bytestring(text):
+        # py2k only
+        if not isinstance(text, string_types):
+            return unicode(text).encode("ascii", errors="backslashreplace")
+        elif isinstance(text, unicode):
+            return text.encode("ascii", errors="backslashreplace")
+        else:
+            return text
+
    # not as nice as that of Py3K, but at least preserves
    # the code line where the issue occurred
    exec(
@@ -1649,6 +1649,14 @@ def inject_param_text(doctext, inject_params):
            lines.append(to_inject)
            lines.append("\n")
            to_inject = None
+        elif line.endswith("::"):
+            # TODO: this still wont cover if the code example itself has blank
+            # lines in it, need to detect those via indentation.
+            lines.append(line)
+            lines.append(
+                doclines.pop(0)
+            )  # the blank line following a code example
+            continue
        lines.append(line)

    return "\n".join(lines)
@@ -70,9 +70,26 @@ class WrapTest(fixtures.TestBase):
        except sa_exceptions.DBAPIError as exc:
            eq_(
                str(exc),
-                "(test.base.test_except.OperationalError)  "
-                "[SQL: 'this is a message'] (Background on this error at: "
-                "http://sqlalche.me/e/e3q8)",
+                "(test.base.test_except.OperationalError) \n"
+                "[SQL: this is a message]\n"
+                "(Background on this error at: http://sqlalche.me/e/e3q8)",
+            )
+
+    def test_tostring_with_newlines(self):
+        try:
+            raise sa_exceptions.DBAPIError.instance(
+                "this is a message\nthis is the next line\nthe last line",
+                None,
+                OperationalError(),
+                DatabaseError,
+            )
+        except sa_exceptions.DBAPIError as exc:
+            eq_(
+                str(exc),
+                "(test.base.test_except.OperationalError) \n"
+                "[SQL: this is a message\nthis is the next line\n"
+                "the last line]\n"
+                "(Background on this error at: http://sqlalche.me/e/e3q8)",
            )

    def test_statement_error_no_code(self):
@@ -86,8 +103,8 @@ class WrapTest(fixtures.TestBase):
        except sa_exceptions.StatementError as err:
            eq_(
                str(err),
-                "(sqlalchemy.exc.InvalidRequestError) hello "
-                "[SQL: 'select * from table'] [parameters: [{'x': 1}]]",
+                "(sqlalchemy.exc.InvalidRequestError) hello\n"
+                "[SQL: select * from table]\n[parameters: [{'x': 1}]]",
            )
            eq_(err.args, ("(sqlalchemy.exc.InvalidRequestError) hello",))

@@ -102,8 +119,9 @@ class WrapTest(fixtures.TestBase):
        except sa_exceptions.StatementError as err:
            eq_(
                str(err),
-                "(sqlalchemy.exc.InvalidRequestError) hello "
-                "[SQL: 'select * from table'] [parameters: [{'x': 1}]] "
+                "(sqlalchemy.exc.InvalidRequestError) hello\n"
+                "[SQL: select * from table]\n"
+                "[parameters: [{'x': 1}]]\n"
                "(Background on this error at: http://sqlalche.me/e/abcd)",
            )
            eq_(err.args, ("(sqlalchemy.exc.InvalidRequestError) hello",))
@@ -114,7 +132,7 @@ class WrapTest(fixtures.TestBase):
        orig.args = [2006, "Test raise operational error"]
        eq_(
            str(orig),
-            "(2006, 'Test raise operational error') "
+            "(2006, 'Test raise operational error')\n"
            "(Background on this error at: http://sqlalche.me/e/dbapi)",
        )

@@ -125,7 +143,7 @@ class WrapTest(fixtures.TestBase):
        eq_(
            compat.text_type(orig),
            compat.u(
-                "méil (Background on this error at: "
+                "méil\n(Background on this error at: "
                "http://sqlalche.me/e/dbapi)"
            ),
        )
@@ -153,8 +171,9 @@ class WrapTest(fixtures.TestBase):
            )
        except sa_exceptions.DBAPIError as exc:
            assert str(exc).startswith(
-                "(test.base.test_except.OperationalError)  "
-                "[SQL: 'this is a message'] [parameters: {"
+                "(test.base.test_except.OperationalError) \n"
+                "[SQL: this is a message]\n"
+                "[parameters: {"
            )

    def test_tostring_large_list(self):
@@ -165,10 +184,10 @@ class WrapTest(fixtures.TestBase):
                OperationalError(),
                DatabaseError,
            )
-        except sa_exceptions.DBAPIError as exc:
-            assert str(exc).startswith(
-                "(test.base.test_except.OperationalError)  "
-                "[SQL: 'this is a message'] [parameters: "
+        except sa_exceptions.DBAPIError as ex:
+            assert str(ex).startswith(
+                "(test.base.test_except.OperationalError) \n"
+                "[SQL: this is a message]\n[parameters: "
                "[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]]"
            )

@@ -194,11 +213,11 @@ class WrapTest(fixtures.TestBase):
        except sa_exceptions.DBAPIError as exc:
            eq_(
                str(exc),
-                "(test.base.test_except.OperationalError) sql error "
-                "[SQL: 'this is a message'] [parameters: [{1: 1}, "
-                "{1: 1}, {1: 1}, {1: 1}, {1: 1}, {1: 1}, {1: 1}, {1: "
-                "1}, {1: 1}, {1: 1}]] (Background on this error at: "
-                "http://sqlalche.me/e/e3q8)",
+                "(test.base.test_except.OperationalError) sql error\n"
+                "[SQL: this is a message]\n"
+                "[parameters: [{1: 1}, {1: 1}, {1: 1}, {1: 1}, {1: 1},"
+                " {1: 1}, {1: 1}, {1: 1}, {1: 1}, {1: 1}]]\n"
+                "(Background on this error at: http://sqlalche.me/e/e3q8)",
            )
            eq_(
                exc.args,
@@ -226,11 +245,12 @@ class WrapTest(fixtures.TestBase):
        except sa_exceptions.DBAPIError as exc:
            eq_(
                str(exc),
-                "(test.base.test_except.OperationalError)  "
-                "[SQL: 'this is a message'] [parameters: [{1: 1}, "
+                "(test.base.test_except.OperationalError) \n"
+                "[SQL: this is a message]\n"
+                "[parameters: [{1: 1}, "
                "{1: 1}, {1: 1}, {1: 1}, {1: 1}, {1: 1}, "
                "{1: 1}, {1: 1}  ... displaying 10 of 11 total "
-                "bound parameter sets ...  {1: 1}, {1: 1}]] "
+                "bound parameter sets ...  {1: 1}, {1: 1}]]\n"
                "(Background on this error at: http://sqlalche.me/e/e3q8)",
            )
        try:
@@ -244,9 +264,10 @@ class WrapTest(fixtures.TestBase):
        except sa_exceptions.DBAPIError as exc:
            eq_(
                str(exc),
-                "(test.base.test_except.OperationalError)  "
-                "[SQL: 'this is a message'] [parameters: [(1,), "
-                "(1,), (1,), (1,), (1,), (1,), (1,), (1,), (1,), (1,)]] "
+                "(test.base.test_except.OperationalError) \n"
+                "[SQL: this is a message]\n"
+                "[parameters: [(1,), "
+                "(1,), (1,), (1,), (1,), (1,), (1,), (1,), (1,), (1,)]]\n"
                "(Background on this error at: http://sqlalche.me/e/e3q8)",
            )
        try:
@@ -271,11 +292,12 @@ class WrapTest(fixtures.TestBase):
        except sa_exceptions.DBAPIError as exc:
            eq_(
                str(exc),
-                "(test.base.test_except.OperationalError)  "
-                "[SQL: 'this is a message'] [parameters: [(1,), "
+                "(test.base.test_except.OperationalError) \n"
+                "[SQL: this is a message]\n"
+                "[parameters: [(1,), "
                "(1,), (1,), (1,), (1,), (1,), (1,), (1,)  "
                "... displaying 10 of 11 total bound "
-                "parameter sets ...  (1,), (1,)]] "
+                "parameter sets ...  (1,), (1,)]]\n"
                "(Background on this error at: http://sqlalche.me/e/e3q8)",
            )

@@ -2305,6 +2305,10 @@ class AsInterfaceTest(fixtures.TestBase):
    class Object(object):
        pass

+    def test_no_cls_no_methods(self):
+        obj = object()
+        assert_raises(TypeError, util.as_interface, obj)
+
    def test_instance(self):
        obj = object()
        assert_raises(TypeError, util.as_interface, obj, cls=self.Something)
@@ -435,6 +435,40 @@ class VersionDetectionTest(fixtures.TestBase):
            eq_(dialect._get_server_version_info(conn), expected)


+class RealIsolationLevelTest(fixtures.TestBase):
+    __only_on__ = "mssql"
+    __backend__ = True
+
+    @testing.provide_metadata
+    def test_isolation_level(self):
+        Table("test", self.metadata, Column("id", Integer)).create(
+            checkfirst=True
+        )
+
+        with testing.db.connect() as c:
+            default = testing.db.dialect.get_isolation_level(c.connection)
+
+        values = [
+            "READ UNCOMMITTED",
+            "READ COMMITTED",
+            "REPEATABLE READ",
+            "SERIALIZABLE",
+            "SNAPSHOT",
+        ]
+        for value in values:
+            with testing.db.connect() as c:
+                c.execution_options(isolation_level=value)
+
+                c.execute("SELECT TOP 10 * FROM test")
+
+                eq_(
+                    testing.db.dialect.get_isolation_level(c.connection), value
+                )
+
+        with testing.db.connect() as c:
+            eq_(testing.db.dialect.get_isolation_level(c.connection), default)
+
+
 class IsolationLevelDetectTest(fixtures.TestBase):
    def _fixture(self, view):
        class Error(Exception):
@@ -7,6 +7,7 @@ from sqlalchemy import event
 from sqlalchemy import ForeignKey
 from sqlalchemy import func
 from sqlalchemy import Integer
+from sqlalchemy import literal
 from sqlalchemy import MetaData
 from sqlalchemy import or_
 from sqlalchemy import PrimaryKeyConstraint
@@ -172,36 +173,68 @@ class IdentityInsertTest(fixtures.TestBase, AssertsCompiledSQL):
        )

    def test_execute(self):
-        cattable.insert().values(id=9, description="Python").execute()
+        with testing.db.connect() as conn:
+            conn.execute(cattable.insert().values(id=9, description="Python"))

-        cats = cattable.select().order_by(cattable.c.id).execute()
-        eq_([(9, "Python")], list(cats))
+            cats = conn.execute(cattable.select().order_by(cattable.c.id))
+            eq_([(9, "Python")], list(cats))

-        result = cattable.insert().values(description="PHP").execute()
-        eq_([10], result.inserted_primary_key)
-        lastcat = cattable.select().order_by(desc(cattable.c.id)).execute()
-        eq_((10, "PHP"), lastcat.first())
+            result = conn.execute(cattable.insert().values(description="PHP"))
+            eq_([10], result.inserted_primary_key)
+            lastcat = conn.execute(
+                cattable.select().order_by(desc(cattable.c.id))
+            )
+            eq_((10, "PHP"), lastcat.first())

    def test_executemany(self):
-        cattable.insert().execute(
-            [
-                {"id": 89, "description": "Python"},
-                {"id": 8, "description": "Ruby"},
-                {"id": 3, "description": "Perl"},
-                {"id": 1, "description": "Java"},
-            ]
-        )
-        cats = cattable.select().order_by(cattable.c.id).execute()
-        eq_(
-            [(1, "Java"), (3, "Perl"), (8, "Ruby"), (89, "Python")], list(cats)
-        )
-        cattable.insert().execute(
-            [{"description": "PHP"}, {"description": "Smalltalk"}]
-        )
-        lastcats = (
-            cattable.select().order_by(desc(cattable.c.id)).limit(2).execute()
-        )
-        eq_([(91, "Smalltalk"), (90, "PHP")], list(lastcats))
+        with testing.db.connect() as conn:
+            conn.execute(
+                cattable.insert(),
+                [
+                    {"id": 89, "description": "Python"},
+                    {"id": 8, "description": "Ruby"},
+                    {"id": 3, "description": "Perl"},
+                    {"id": 1, "description": "Java"},
+                ],
+            )
+            cats = conn.execute(cattable.select().order_by(cattable.c.id))
+            eq_(
+                [(1, "Java"), (3, "Perl"), (8, "Ruby"), (89, "Python")],
+                list(cats),
+            )
+            conn.execute(
+                cattable.insert(),
+                [{"description": "PHP"}, {"description": "Smalltalk"}],
+            )
+            lastcats = conn.execute(
+                cattable.select().order_by(desc(cattable.c.id)).limit(2)
+            )
+            eq_([(91, "Smalltalk"), (90, "PHP")], list(lastcats))
+
+    def test_insert_plain_param(self):
+        with testing.db.connect() as conn:
+            conn.execute(cattable.insert(), id=5)
+            eq_(conn.scalar(select([cattable.c.id])), 5)
+
+    def test_insert_values_key_plain(self):
+        with testing.db.connect() as conn:
+            conn.execute(cattable.insert().values(id=5))
+            eq_(conn.scalar(select([cattable.c.id])), 5)
+
+    def test_insert_values_key_expression(self):
+        with testing.db.connect() as conn:
+            conn.execute(cattable.insert().values(id=literal(5)))
+            eq_(conn.scalar(select([cattable.c.id])), 5)
+
+    def test_insert_values_col_plain(self):
+        with testing.db.connect() as conn:
+            conn.execute(cattable.insert().values({cattable.c.id: 5}))
+            eq_(conn.scalar(select([cattable.c.id])), 5)
+
+    def test_insert_values_col_expression(self):
+        with testing.db.connect() as conn:
+            conn.execute(cattable.insert().values({cattable.c.id: literal(5)}))
+            eq_(conn.scalar(select([cattable.c.id])), 5)


 class QueryUnicodeTest(fixtures.TestBase):
@@ -42,8 +42,10 @@ class ReflectionTest(fixtures.TestBase, ComparesTables, AssertsCompiledSQL):
            Column("user_name", types.VARCHAR(20), nullable=False),
            Column("test1", types.CHAR(5), nullable=False),
            Column("test2", types.Float(5), nullable=False),
+            Column("test2.5", types.Float(), nullable=False),
            Column("test3", types.Text()),
            Column("test4", types.Numeric, nullable=False),
+            Column("test4.5", types.Numeric(10, 2), nullable=False),
            Column("test5", types.DateTime),
            Column(
                "parent_user_id",
@@ -105,6 +107,19 @@ class ReflectionTest(fixtures.TestBase, ComparesTables, AssertsCompiledSQL):
    def test_money_type(self):
        self._test_specific_type(mssql.MONEY, "MONEY")

+    def test_numeric_prec_scale(self):
+        self._test_specific_type(mssql.NUMERIC(10, 2), "NUMERIC(10, 2)")
+
+    def test_float(self):
+        self._test_specific_type(mssql.FLOAT, "FLOAT(53)")
+
+    def test_real(self):
+        self._test_specific_type(mssql.REAL, "REAL")
+
+    def test_float_as_real(self):
+        # FLOAT(5) comes back as REAL
+        self._test_specific_type(mssql.FLOAT(5), "REAL")
+
    @testing.provide_metadata
    def test_identity(self):
        metadata = self.metadata
@@ -27,6 +27,7 @@ from sqlalchemy import TIMESTAMP
 from sqlalchemy import Unicode
 from sqlalchemy import UnicodeText
 from sqlalchemy import UniqueConstraint
+from sqlalchemy import util
 from sqlalchemy.dialects.mysql import base as mysql
 from sqlalchemy.dialects.mysql import reflection as _reflection
 from sqlalchemy.schema import CreateIndex
@@ -36,6 +37,7 @@ from sqlalchemy.testing import eq_
 from sqlalchemy.testing import expect_warnings
 from sqlalchemy.testing import fixtures
 from sqlalchemy.testing import is_
+from sqlalchemy.testing import mock


 class TypeReflectionTest(fixtures.TestBase):
@@ -757,6 +759,117 @@ class ReflectionTest(fixtures.TestBase, AssertsCompiledSQL):
            [{"name": "foo_idx", "column_names": ["x"], "unique": False}],
        )

+    def _bug_88718_casing_0(self):
+        fkeys_casing_0 = [
+            {
+                "name": "FK_PlaylistTTrackId",
+                "constrained_columns": ["TTrackID"],
+                "referred_schema": "test_schema",
+                "referred_table": "Track",
+                "referred_columns": ["trackid"],
+                "options": {},
+            },
+            {
+                "name": "FK_PlaylistTrackId",
+                "constrained_columns": ["TrackID"],
+                "referred_schema": None,
+                "referred_table": "Track",
+                "referred_columns": ["trackid"],
+                "options": {},
+            },
+        ]
+        ischema_casing_0 = [
+            ("test", "Track", "TrackID"),
+            ("test_schema", "Track", "TrackID"),
+        ]
+        return fkeys_casing_0, ischema_casing_0
+
+    def _bug_88718_casing_1(self):
+        fkeys_casing_1 = [
+            {
+                "name": "FK_PlaylistTTrackId",
+                "constrained_columns": ["TTrackID"],
+                "referred_schema": "test_schema",
+                "referred_table": "Track",
+                "referred_columns": ["trackid"],
+                "options": {},
+            },
+            {
+                "name": "FK_PlaylistTrackId",
+                "constrained_columns": ["TrackID"],
+                "referred_schema": None,
+                "referred_table": "Track",
+                "referred_columns": ["trackid"],
+                "options": {},
+            },
+        ]
+        ischema_casing_1 = [
+            (util.u("test"), util.u("Track"), "TrackID"),
+            (util.u("test_schema"), util.u("Track"), "TrackID"),
+        ]
+        return fkeys_casing_1, ischema_casing_1
+
+    def _bug_88718_casing_2(self):
+        fkeys_casing_2 = [
+            {
+                "name": "FK_PlaylistTTrackId",
+                "constrained_columns": ["TTrackID"],
+                "referred_schema": "test_schema",
+                "referred_table": "Track",
+                "referred_columns": ["trackid"],
+                "options": {},
+            },
+            {
+                "name": "FK_PlaylistTrackId",
+                "constrained_columns": ["TrackID"],
+                "referred_schema": None,
+                "referred_table": "Track",
+                "referred_columns": ["trackid"],
+                "options": {},
+            },
+        ]
+        ischema_casing_2 = [
+            ("test", "Track", "TrackID"),
+            ("test_schema", "Track", "TrackID"),
+        ]
+        return fkeys_casing_2, ischema_casing_2
+
+    def test_correct_for_mysql_bug_88718(self):
+        dialect = mysql.dialect()
+
+        for casing, (fkeys, ischema) in [
+            (0, self._bug_88718_casing_0()),
+            (1, self._bug_88718_casing_1()),
+            (2, self._bug_88718_casing_2()),
+        ]:
+            dialect._casing = casing
+            dialect.default_schema_name = "test"
+            connection = mock.Mock(
+                dialect=dialect, execute=lambda stmt, **params: ischema
+            )
+            dialect._correct_for_mysql_bug_88718(fkeys, connection)
+            eq_(
+                fkeys,
+                [
+                    {
+                        "name": "FK_PlaylistTTrackId",
+                        "constrained_columns": ["TTrackID"],
+                        "referred_schema": "test_schema",
+                        "referred_table": "Track",
+                        "referred_columns": ["TrackID"],
+                        "options": {},
+                    },
+                    {
+                        "name": "FK_PlaylistTrackId",
+                        "constrained_columns": ["TrackID"],
+                        "referred_schema": None,
+                        "referred_table": "Track",
+                        "referred_columns": ["TrackID"],
+                        "options": {},
+                    },
+                ],
+            )
+
    @testing.provide_metadata
    def test_case_sensitive_column_constraint_reflection(self):
        # test for issue #4344 which works around
@@ -686,22 +686,23 @@ class TypesTest(
        # server-system-variables.html#sysvar_explicit_defaults_for_timestamp
        # changes this for 5.6 if set.

-        # normalize dates that are over the second boundary
+        # normalize dates for the amount of time the operation took
        def normalize(dt):
            if dt is None:
                return None
-            elif (dt - now).seconds < 5:
+            elif now <= dt <= new_now:
                return now
            else:
                return dt

        with testing.db.begin() as conn:
            now = conn.scalar("select now()")
-
            conn.execute(ts_table.insert(), {"t1": now, "t2": None})
            conn.execute(ts_table.insert(), {"t1": None, "t2": None})
            conn.execute(ts_table.insert(), {"t2": None})

+            new_now = conn.scalar("select now()")
+
            eq_(
                [
                    tuple([normalize(dt) for dt in row])
@@ -782,23 +782,37 @@ class TypesTest(fixtures.TestBase):
    @testing.provide_metadata
    def test_reflect_nvarchar(self):
        metadata = self.metadata
-        Table("tnv", metadata, Column("data", sqltypes.NVARCHAR(255)))
+        Table(
+            "tnv",
+            metadata,
+            Column("nv_data", sqltypes.NVARCHAR(255)),
+            Column("c_data", sqltypes.NCHAR(20)),
+        )
        metadata.create_all()
        m2 = MetaData(testing.db)
        t2 = Table("tnv", m2, autoload=True)
-        assert isinstance(t2.c.data.type, sqltypes.NVARCHAR)
+        assert isinstance(t2.c.nv_data.type, sqltypes.NVARCHAR)
+        assert isinstance(t2.c.c_data.type, sqltypes.NCHAR)

        if testing.against("oracle+cx_oracle"):
            assert isinstance(
-                t2.c.data.type.dialect_impl(testing.db.dialect),
+                t2.c.nv_data.type.dialect_impl(testing.db.dialect),
+                cx_oracle._OracleUnicodeStringNCHAR,
+            )
+
+            assert isinstance(
+                t2.c.c_data.type.dialect_impl(testing.db.dialect),
                cx_oracle._OracleUnicodeStringNCHAR,
            )

        data = u("m’a réveillé.")
-        t2.insert().execute(data=data)
-        res = t2.select().execute().first()["data"]
-        eq_(res, data)
-        assert isinstance(res, util.text_type)
+        with testing.db.connect() as conn:
+            conn.execute(t2.insert(), dict(nv_data=data, c_data=data))
+            nv_data, c_data = conn.execute(t2.select()).first()
+            eq_(nv_data, data)
+            eq_(c_data, data + (" " * 7))  # char is space padded
+            assert isinstance(nv_data, util.text_type)
+            assert isinstance(c_data, util.text_type)

    @testing.provide_metadata
    def test_reflect_unicode_no_nvarchar(self):
@@ -830,6 +844,7 @@ class TypesTest(fixtures.TestBase):
            Column("c1", VARCHAR(50)),
            Column("c2", NVARCHAR(250)),
            Column("c3", CHAR(200)),
+            Column("c4", NCHAR(180)),
        )
        t1.create()
        m2 = MetaData(testing.db)
@@ -837,6 +852,7 @@ class TypesTest(fixtures.TestBase):
        eq_(t2.c.c1.type.length, 50)
        eq_(t2.c.c2.type.length, 250)
        eq_(t2.c.c3.type.length, 200)
+        eq_(t2.c.c4.type.length, 180)

    @testing.provide_metadata
    def test_long_type(self):
@@ -1056,6 +1056,30 @@ class CompileTest(fixtures.TestBase, AssertsCompiledSQL):
            "WHERE mytable_1.myid = %(myid_1)s FOR UPDATE OF mytable_1",
        )

+        table2 = table("table2", column("mytable_id"))
+        join = table2.join(table1, table2.c.mytable_id == table1.c.myid)
+        self.assert_compile(
+            join.select(table2.c.mytable_id == 7).with_for_update(of=[join]),
+            "SELECT table2.mytable_id, "
+            "mytable.myid, mytable.name, mytable.description "
+            "FROM table2 "
+            "JOIN mytable ON table2.mytable_id = mytable.myid "
+            "WHERE table2.mytable_id = %(mytable_id_1)s "
+            "FOR UPDATE OF mytable, table2",
+        )
+
+        join = table2.join(ta, table2.c.mytable_id == ta.c.myid)
+        self.assert_compile(
+            join.select(table2.c.mytable_id == 7).with_for_update(of=[join]),
+            "SELECT table2.mytable_id, "
+            "mytable_1.myid, mytable_1.name, mytable_1.description "
+            "FROM table2 "
+            "JOIN mytable AS mytable_1 "
+            "ON table2.mytable_id = mytable_1.myid "
+            "WHERE table2.mytable_id = %(mytable_id_1)s "
+            "FOR UPDATE OF mytable_1, table2",
+        )
+
    def test_for_update_with_schema(self):
        m = MetaData()
        table1 = Table(
@@ -1114,7 +1114,7 @@ class HandleErrorTest(fixtures.TestBase):
        with engine.connect() as conn:
            assert_raises_message(
                tsa.exc.StatementError,
-                r"\(.*SomeException\) " r"nope \[SQL\: u?'SELECT 1 ",
+                r"\(.*SomeException\) " r"nope\n\[SQL\: u?SELECT 1 ",
                conn.execute,
                select([1]).where(column("foo") == literal("bar", MyType())),
            )
@@ -371,7 +371,7 @@ class ExecuteTest(fixtures.TestBase):
        def _go(conn):
            assert_raises_message(
                tsa.exc.StatementError,
-                r"\(.*.SomeException\) " r"nope \[SQL\: u?'SELECT 1 ",
+                r"\(.*.SomeException\) " r"nope\n\[SQL\: u?SELECT 1 ",
                conn.execute,
                select([1]).where(column("foo") == literal("bar", MyType())),
            )
@@ -410,12 +410,12 @@ class ExecuteTest(fixtures.TestBase):
            assert_raises_message(
                tsa.exc.StatementError,
                util.u(
-                    "A value is required for bind parameter 'uname'"
-                    r".*SELECT users.user_name AS .m\\xe9il."
+                    "A value is required for bind parameter 'uname'\n"
+                    r".*SELECT users.user_name AS .m\xe9il."
                )
                if util.py2k
                else util.u(
-                    "A value is required for bind parameter 'uname'"
+                    "A value is required for bind parameter 'uname'\n"
                    ".*SELECT users.user_name AS .méil."
                ),
                conn.execute,
@@ -2184,7 +2184,7 @@ class HandleErrorTest(fixtures.TestBase):
        with engine.connect() as conn:
            assert_raises_message(
                tsa.exc.StatementError,
-                r"\(.*.SomeException\) " r"nope \[SQL\: u?'SELECT 1 ",
+                r"\(.*.SomeException\) " r"nope\n\[SQL\: u?SELECT 1 ",
                conn.execute,
                select([1]).where(column("foo") == literal("bar", MyType())),
            )
@@ -103,7 +103,7 @@ class LogParamsTest(fixtures.TestBase):
        exception = tsa.exc.IntegrityError("foo", {"x": "y"}, None)
        eq_regex(
            str(exception),
-            r"\(.*.NoneType\) None \[SQL: 'foo'\] \[parameters: {'x': 'y'}\]",
+            r"\(.*.NoneType\) None\n\[SQL: foo\]\n\[parameters: {'x': 'y'}\]",
        )

    def test_exception_format_unexpected_parameter(self):
@@ -112,7 +112,7 @@ class LogParamsTest(fixtures.TestBase):
        exception = tsa.exc.IntegrityError("foo", "bar", "bat")
        eq_regex(
            str(exception),
-            r"\(.*.str\) bat \[SQL: 'foo'\] \[parameters: 'bar'\]",
+            r"\(.*.str\) bat\n\[SQL: foo\]\n\[parameters: 'bar'\]",
        )

    def test_exception_format_unexpected_member_parameter(self):
@@ -121,7 +121,7 @@ class LogParamsTest(fixtures.TestBase):
        exception = tsa.exc.IntegrityError("foo", ["bar", "bat"], "hoho")
        eq_regex(
            str(exception),
-            r"\(.*.str\) hoho \[SQL: 'foo'\] \[parameters: \['bar', 'bat'\]\]",
+            r"\(.*.str\) hoho\n\[SQL: foo\]\n\[parameters: \['bar', 'bat'\]\]",
        )

    def test_result_large_param(self):
@@ -169,7 +169,7 @@ class LogParamsTest(fixtures.TestBase):
    def test_error_large_dict(self):
        assert_raises_message(
            tsa.exc.DBAPIError,
-            r".*'INSERT INTO nonexistent \(data\) values \(:data\)'\] "
+            r".*INSERT INTO nonexistent \(data\) values \(:data\)\]\n"
            r"\[parameters: "
            r"\[{'data': '0'}, {'data': '1'}, {'data': '2'}, "
            r"{'data': '3'}, {'data': '4'}, {'data': '5'}, "
@@ -186,7 +186,7 @@ class LogParamsTest(fixtures.TestBase):
        assert_raises_message(
            tsa.exc.DBAPIError,
            r".*INSERT INTO nonexistent \(data\) values "
-            r"\(\?\)'\] \[parameters: \[\('0',\), \('1',\), \('2',\), "
+            r"\(\?\)\]\n\[parameters: \[\('0',\), \('1',\), \('2',\), "
            r"\('3',\), \('4',\), \('5',\), \('6',\), \('7',\)  "
            r"... displaying "
            r"10 of 100 total bound parameter sets ...  "
@@ -12,6 +12,8 @@ from sqlalchemy.testing import assert_raises
 from sqlalchemy.testing import eq_
 from sqlalchemy.testing import fixtures
 from sqlalchemy.testing import is_
+from sqlalchemy.testing import is_false
+from sqlalchemy.testing import is_true
 from sqlalchemy.testing import mock
 from sqlalchemy.testing.mock import call
 from sqlalchemy.testing.mock import MagicMock
@@ -21,7 +23,7 @@ from sqlalchemy.testing.mock import Mock
 dialect = None


-class ParseConnectTest(fixtures.TestBase):
+class URLTest(fixtures.TestBase):
    def test_rfc1738(self):
        for text in (
            "dbtype://username:password@hostspec:110//usr/db_file.db",
@@ -145,6 +147,38 @@ class ParseConnectTest(fixtures.TestBase):
            "dialect://user:pass@host/db?arg1=param1&arg2=param2&arg2=param3",
        )

+    def test_comparison(self):
+        components = (
+            "drivername",
+            "username",
+            "password",
+            "host",
+            "database",
+            "query",
+            "port",
+        )
+
+        common_url = (
+            "dbtype://username:password"
+            "@[2001:da8:2004:1000:202:116:160:90]:80/database?foo=bar"
+        )
+        other_url = "dbtype://uname:pwd@host/"
+
+        url1 = url.make_url(common_url)
+        url2 = url.make_url(common_url)
+        url3 = url.make_url(other_url)
+
+        is_true(url1 == url2)
+        is_false(url1 != url2)
+        is_true(url1 != url3)
+        is_false(url1 == url3)
+
+        for curr_component in components:
+            setattr(url2, curr_component, "new_changed_value")
+            is_true(url1 != url2)
+            is_false(url1 == url2)
+            setattr(url2, curr_component, getattr(url1, curr_component))
+

 class DialectImportTest(fixtures.TestBase):
    def test_import_base_dialects(self):
@@ -15,6 +15,7 @@ from sqlalchemy.testing import eq_
 from sqlalchemy.testing import fixtures
 from sqlalchemy.testing import is_
 from sqlalchemy.testing import is_not_
+from sqlalchemy.testing import is_true
 from sqlalchemy.testing.engines import testing_engine
 from sqlalchemy.testing.mock import ANY
 from sqlalchemy.testing.mock import call
@@ -776,8 +777,7 @@ class QueuePoolTest(PoolTestBase):

    def test_timeout_accessor(self):
        expected_timeout = 123
-        p = self._queuepool_fixture(
-            timeout=expected_timeout)
+        p = self._queuepool_fixture(timeout=expected_timeout)
        eq_(p.timeout(), expected_timeout)

    @testing.requires.timing_intensive
@@ -1810,7 +1810,9 @@ class SingletonThreadPoolTest(PoolTestBase):
            threads.append(th)
        for th in threads:
            th.join(join_timeout)
-        eq_(len(p._all_conns), 3)
+
+        lp = len(p._all_conns)
+        is_true(3 <= lp <= 4)

        if strong_refs:
            still_opened = len([c for c in sr if not c.close.call_count])
@@ -23,8 +23,10 @@ from sqlalchemy.testing import eq_
 from sqlalchemy.testing import eq_regex
 from sqlalchemy.testing import expect_warnings
 from sqlalchemy.testing import fixtures
+from sqlalchemy.testing import in_
 from sqlalchemy.testing import is_true
 from sqlalchemy.testing import mock
+from sqlalchemy.testing import not_in_
 from sqlalchemy.testing import skip
 from sqlalchemy.testing.schema import Column
 from sqlalchemy.testing.schema import Table
@@ -142,6 +144,112 @@ class ReflectionTest(fixtures.TestBase, ComparesTables):
        assert t1r.c.t2id.references(t2r.c.id)
        assert t1r.c.t3id.references(t3r.c.id)

+    @testing.provide_metadata
+    def test_resolve_fks_false_table(self):
+        meta = self.metadata
+        Table(
+            "t1",
+            meta,
+            Column("id", sa.Integer, primary_key=True),
+            Column("t2id", sa.Integer, sa.ForeignKey("t2.id")),
+            test_needs_fk=True,
+        )
+        Table(
+            "t2",
+            meta,
+            Column("id", sa.Integer, primary_key=True),
+            test_needs_fk=True,
+        )
+        meta.create_all()
+        meta2 = MetaData()
+        t1 = Table("t1", meta2, resolve_fks=False, autoload_with=testing.db)
+        in_("t1", meta2.tables)
+        not_in_("t2", meta2.tables)
+
+        assert_raises(
+            sa.exc.NoReferencedTableError,
+            lambda: list(t1.c.t2id.foreign_keys)[0].column,
+        )
+
+        t2 = Table("t2", meta2, autoload_with=testing.db)
+
+        # now it resolves
+        is_true(t1.c.t2id.references(t2.c.id))
+
+    @testing.provide_metadata
+    def test_resolve_fks_false_extend_existing(self):
+        meta = self.metadata
+        Table(
+            "t1",
+            meta,
+            Column("id", sa.Integer, primary_key=True),
+            Column("t2id", sa.Integer, sa.ForeignKey("t2.id")),
+            test_needs_fk=True,
+        )
+        Table(
+            "t2",
+            meta,
+            Column("id", sa.Integer, primary_key=True),
+            test_needs_fk=True,
+        )
+        meta.create_all()
+        meta2 = MetaData()
+        Table("t1", meta2)
+        in_("t1", meta2.tables)
+
+        t1 = Table(
+            "t1",
+            meta2,
+            resolve_fks=False,
+            autoload_with=testing.db,
+            extend_existing=True,
+        )
+        not_in_("t2", meta2.tables)
+
+        assert_raises(
+            sa.exc.NoReferencedTableError,
+            lambda: list(t1.c.t2id.foreign_keys)[0].column,
+        )
+
+        t2 = Table("t2", meta2, autoload_with=testing.db)
+
+        # now it resolves
+        is_true(t1.c.t2id.references(t2.c.id))
+
+    @testing.provide_metadata
+    def test_resolve_fks_false_metadata(self):
+        meta = self.metadata
+        Table(
+            "t1",
+            meta,
+            Column("id", sa.Integer, primary_key=True),
+            Column("t2id", sa.Integer, sa.ForeignKey("t2.id")),
+            test_needs_fk=True,
+        )
+        Table(
+            "t2",
+            meta,
+            Column("id", sa.Integer, primary_key=True),
+            test_needs_fk=True,
+        )
+        meta.create_all()
+        meta2 = MetaData()
+        meta2.reflect(testing.db, resolve_fks=False, only=["t1"])
+        in_("t1", meta2.tables)
+        not_in_("t2", meta2.tables)
+
+        t1 = meta2.tables["t1"]
+
+        assert_raises(
+            sa.exc.NoReferencedTableError,
+            lambda: list(t1.c.t2id.foreign_keys)[0].column,
+        )
+
+        meta2.reflect(testing.db, resolve_fks=False)
+
+        t2 = meta2.tables["t2"]
+        is_true(t1.c.t2id.references(t2.c.id))
+
    def test_nonexistent(self):
        meta = MetaData(testing.db)
        assert_raises(
@@ -3089,6 +3089,58 @@ class MultiOwnerTest(
        self._assert_raises_ambiguous(lambda: D.c_data.any(B.id == 5))


+class ProxyOfSynonymTest(AssertsCompiledSQL, fixtures.DeclarativeMappedTest):
+    __dialect__ = "default"
+
+    run_create_tables = None
+
+    @classmethod
+    def setup_classes(cls):
+        from sqlalchemy.orm import synonym
+
+        Base = cls.DeclarativeBasic
+
+        class A(Base):
+            __tablename__ = "a"
+
+            id = Column(Integer, primary_key=True)
+            data = Column(String)
+            bs = relationship("B", backref="a")
+            data_syn = synonym("data")
+
+            b_data = association_proxy("bs", "data_syn")
+
+        class B(Base):
+            __tablename__ = "b"
+            id = Column(Integer, primary_key=True)
+            a_id = Column(ForeignKey("a.id"))
+            data = Column(String)
+            data_syn = synonym("data")
+
+            a_data = association_proxy("a", "data_syn")
+
+    def test_o2m_instance_getter(self):
+        A, B = self.classes("A", "B")
+
+        a1 = A(bs=[B(data="bdata1"), B(data="bdata2")])
+        eq_(a1.b_data, ["bdata1", "bdata2"])
+
+    def test_m2o_instance_getter(self):
+        A, B = self.classes("A", "B")
+
+        b1 = B(a=A(data="adata"))
+        eq_(b1.a_data, "adata")
+
+    def test_o2m_expr(self):
+        A, B = self.classes("A", "B")
+
+        self.assert_compile(
+            A.b_data == "foo",
+            "EXISTS (SELECT 1 FROM a, b WHERE a.id = b.a_id "
+            "AND b.data = :data_1)",
+        )
+
+
 class ProxyAttrTest(fixtures.DeclarativeMappedTest):
    @classmethod
    def setup_classes(cls):
@@ -1325,6 +1325,21 @@ class LazyLoaderTest(testing.AssertsCompiledSQL, BakedTest):
            ),
        )

+    def test_simple_lazy_clause_no_race_on_generate(self):
+        User, Address = self._o2m_fixture()
+
+        expr1, paramdict1 = (
+            User.addresses.property._lazy_strategy._simple_lazy_clause
+        )
+
+        # delete the attr, as though a concurrent thread is also generating it
+        del User.addresses.property._lazy_strategy._simple_lazy_clause
+        expr2, paramdict2 = (
+            User.addresses.property._lazy_strategy._simple_lazy_clause
+        )
+
+        eq_(paramdict1, paramdict2)
+
    # additional tests:
    # 1. m2m w lazyload
    # 2. o2m lazyload where m2o backrefs have an eager load, test
@@ -16,6 +16,8 @@ from sqlalchemy import Unicode
 from sqlalchemy import util
 from sqlalchemy.orm import class_mapper
 from sqlalchemy.orm import clear_mappers
+from sqlalchemy.orm import column_property
+from sqlalchemy.orm import contains_eager
 from sqlalchemy.orm import create_session
 from sqlalchemy.orm import join
 from sqlalchemy.orm import joinedload
@@ -24,6 +26,7 @@ from sqlalchemy.orm import polymorphic_union
 from sqlalchemy.orm import Query
 from sqlalchemy.orm import relationship
 from sqlalchemy.orm import Session
+from sqlalchemy.orm import with_polymorphic
 from sqlalchemy.orm.interfaces import MANYTOONE
 from sqlalchemy.testing import AssertsExecutionResults
 from sqlalchemy.testing import eq_
@@ -503,7 +506,7 @@ for jointype in ["join1", "join2", "join3", "join4"]:
    for data in (True, False):
        _fn = _generate_test(jointype, data)
        setattr(RelationshipTest3, _fn.__name__, _fn)
-del func
+del _fn


 class RelationshipTest4(fixtures.MappedTest):
@@ -2139,3 +2142,111 @@ class ColSubclassTest(
            "(a AS a_1 JOIN b AS b_1 ON a_1.id = b_1.id) "
            "ON a.id = b_1.id WHERE b_1.x = :x_1",
        )
+
+
+class CorrelateExceptWPolyAdaptTest(
+    fixtures.DeclarativeMappedTest, testing.AssertsCompiledSQL
+):
+    # test [ticket:4537]'s test case.
+
+    run_create_tables = run_deletes = None
+    run_setup_classes = run_setup_mappers = run_define_tables = "each"
+    __dialect__ = "default"
+
+    def _fixture(self, use_correlate_except):
+
+        Base = self.DeclarativeBasic
+
+        class Superclass(Base):
+            __tablename__ = "s1"
+            id = Column(Integer, primary_key=True)
+            common_id = Column(ForeignKey("c.id"))
+            common_relationship = relationship(
+                "Common", uselist=False, innerjoin=True, lazy="noload"
+            )
+            discriminator_field = Column(String)
+            __mapper_args__ = {
+                "polymorphic_identity": "superclass",
+                "polymorphic_on": discriminator_field,
+            }
+
+        class Subclass(Superclass):
+            __tablename__ = "s2"
+            id = Column(ForeignKey("s1.id"), primary_key=True)
+            __mapper_args__ = {"polymorphic_identity": "subclass"}
+
+        class Common(Base):
+            __tablename__ = "c"
+            id = Column(Integer, primary_key=True)
+
+            if use_correlate_except:
+                num_superclass = column_property(
+                    select([func.count(Superclass.id)])
+                    .where(Superclass.common_id == id)
+                    .correlate_except(Superclass)
+                )
+
+        if not use_correlate_except:
+            Common.num_superclass = column_property(
+                select([func.count(Superclass.id)])
+                .where(Superclass.common_id == Common.id)
+                .correlate(Common)
+            )
+
+        return Common, Superclass
+
+    def test_poly_query_on_correlate(self):
+        Common, Superclass = self._fixture(False)
+
+        poly = with_polymorphic(Superclass, "*")
+
+        s = Session()
+        q = (
+            s.query(poly)
+            .options(contains_eager(poly.common_relationship))
+            .join(poly.common_relationship)
+            .filter(Common.id == 1)
+        )
+
+        # note the order of c.id, subquery changes based on if we
+        # used correlate or correlate_except; this is only with the
+        # patch in place.   Not sure why this happens.
+        self.assert_compile(
+            q,
+            "SELECT c.id AS c_id, (SELECT count(s1.id) AS count_1 "
+            "FROM s1 LEFT OUTER JOIN s2 ON s1.id = s2.id "
+            "WHERE s1.common_id = c.id) AS anon_1, "
+            "s1.id AS s1_id, "
+            "s1.common_id AS s1_common_id, "
+            "s1.discriminator_field AS s1_discriminator_field, "
+            "s2.id AS s2_id FROM s1 "
+            "LEFT OUTER JOIN s2 ON s1.id = s2.id "
+            "JOIN c ON c.id = s1.common_id WHERE c.id = :id_1",
+        )
+
+    def test_poly_query_on_correlate_except(self):
+        Common, Superclass = self._fixture(True)
+
+        poly = with_polymorphic(Superclass, "*")
+
+        s = Session()
+        q = (
+            s.query(poly)
+            .options(contains_eager(poly.common_relationship))
+            .join(poly.common_relationship)
+            .filter(Common.id == 1)
+        )
+
+        # c.id, subquery are reversed.
+        self.assert_compile(
+            q,
+            "SELECT (SELECT count(s1.id) AS count_1 "
+            "FROM s1 LEFT OUTER JOIN s2 ON s1.id = s2.id "
+            "WHERE s1.common_id = c.id) AS anon_1, "
+            "c.id AS c_id, s1.id AS s1_id, "
+            "s1.common_id AS s1_common_id, "
+            "s1.discriminator_field AS s1_discriminator_field, "
+            "s2.id AS s2_id FROM s1 "
+            "LEFT OUTER JOIN s2 ON s1.id = s2.id "
+            "JOIN c ON c.id = s1.common_id WHERE c.id = :id_1",
+        )
@@ -2004,6 +2004,15 @@ class SessionLifecycleEventsTest(_RemoveListeners, _fixtures.FixtureTest):
            [call.pending_to_persistent(sess, u1), call.flag_checked(u1)],
        )

+        u1.name = "u2"
+        sess.flush()
+
+        # event was not called again
+        eq_(
+            listener.mock_calls,
+            [call.pending_to_persistent(sess, u1), call.flag_checked(u1)],
+        )
+
    def test_pending_to_persistent_del(self):
        sess, User, start_events = self._fixture()

@@ -633,6 +633,44 @@ class RawSelectTest(QueryTest, AssertsCompiledSQL):


 class GetTest(QueryTest):
+    def test_get_composite_pk_keyword_based_no_result(self):
+        CompositePk = self.classes.CompositePk
+
+        s = Session()
+        is_(s.query(CompositePk).get({"i": 100, "j": 100}), None)
+
+    def test_get_composite_pk_keyword_based_result(self):
+        CompositePk = self.classes.CompositePk
+
+        s = Session()
+        one_two = s.query(CompositePk).get({"i": 1, "j": 2})
+        eq_(one_two.i, 1)
+        eq_(one_two.j, 2)
+        eq_(one_two.k, 3)
+
+    def test_get_composite_pk_keyword_based_wrong_keys(self):
+        CompositePk = self.classes.CompositePk
+
+        s = Session()
+        q = s.query(CompositePk)
+        assert_raises(sa_exc.InvalidRequestError, q.get, {"i": 1, "k": 2})
+
+    def test_get_composite_pk_keyword_based_too_few_keys(self):
+        CompositePk = self.classes.CompositePk
+
+        s = Session()
+        q = s.query(CompositePk)
+        assert_raises(sa_exc.InvalidRequestError, q.get, {"i": 1})
+
+    def test_get_composite_pk_keyword_based_too_many_keys(self):
+        CompositePk = self.classes.CompositePk
+
+        s = Session()
+        q = s.query(CompositePk)
+        assert_raises(
+            sa_exc.InvalidRequestError, q.get, {"i": 1, "j": "2", "k": 3}
+        )
+
    def test_get(self):
        User = self.classes.User

@@ -1037,7 +1037,7 @@ class DeferredRelationshipExpressionTest(_fixtures.FixtureTest):
        assert_raises_message(
            sa.exc.StatementError,
            "Can't resolve value for column users.id on object "
-            ".User.*.; the object is detached and the value was expired ",
+            ".User.*.; the object is detached and the value was expired",
            q.one,
        )

@@ -10,6 +10,7 @@ from sqlalchemy import event
 from sqlalchemy import ForeignKey
 from sqlalchemy import func
 from sqlalchemy import Integer
+from sqlalchemy import literal
 from sqlalchemy import literal_column
 from sqlalchemy import select
 from sqlalchemy import String
@@ -478,6 +479,8 @@ class ForeignPKTest(fixtures.MappedTest):


 class ClauseAttributesTest(fixtures.MappedTest):
+    __backend__ = True
+
    @classmethod
    def define_tables(cls, metadata):
        Table(
@@ -499,6 +502,15 @@ class ClauseAttributesTest(fixtures.MappedTest):
            Column("value", Boolean),
        )

+        Table(
+            "pk_t",
+            metadata,
+            Column(
+                "p_id", Integer, key="id", autoincrement=True, primary_key=True
+            ),
+            Column("data", String(30)),
+        )
+
    @classmethod
    def setup_classes(cls):
        class User(cls.Comparable):
@@ -507,12 +519,17 @@ class ClauseAttributesTest(fixtures.MappedTest):
        class HasBoolean(cls.Comparable):
            pass

+        class PkDefault(cls.Comparable):
+            pass
+
    @classmethod
    def setup_mappers(cls):
        User, users_t = cls.classes.User, cls.tables.users_t
        HasBoolean, boolean_t = cls.classes.HasBoolean, cls.tables.boolean_t
+        PkDefault, pk_t = cls.classes.PkDefault, cls.tables.pk_t
        mapper(User, users_t)
        mapper(HasBoolean, boolean_t)
+        mapper(PkDefault, pk_t)

    def test_update(self):
        User = self.classes.User
@@ -568,6 +585,19 @@ class ClauseAttributesTest(fixtures.MappedTest):

        assert (u.counter == 5) is True

+    @testing.requires.sql_expressions_inserted_as_primary_key
+    def test_insert_pk_expression(self):
+        PkDefault = self.classes.PkDefault
+
+        pk = PkDefault(id=literal(5) + 10, data="some data")
+        session = Session()
+        session.add(pk)
+        session.flush()
+
+        eq_(pk.id, 15)
+        session.commit()
+        eq_(pk.id, 15)
+
    def test_update_special_comparator(self):
        HasBoolean = self.classes.HasBoolean

@@ -1,9 +1,11 @@
+from sqlalchemy import cast
 from sqlalchemy import event
 from sqlalchemy import exc
 from sqlalchemy import FetchedValue
 from sqlalchemy import ForeignKey
 from sqlalchemy import func
 from sqlalchemy import Integer
+from sqlalchemy import JSON
 from sqlalchemy import literal
 from sqlalchemy import select
 from sqlalchemy import String
@@ -2669,6 +2671,20 @@ class NullEvaluatingTest(fixtures.MappedTest, testing.AssertsExecutionResults):
            ),
        )

+        if testing.requires.json_type.enabled:
+            Table(
+                "test_has_json",
+                metadata,
+                Column(
+                    "id",
+                    Integer,
+                    primary_key=True,
+                    test_needs_autoincrement=True,
+                ),
+                Column("data", JSON(none_as_null=True).evaluates_none()),
+                Column("data_null", JSON(none_as_null=True)),
+            )
+
    @classmethod
    def setup_classes(cls):
        class Thing(cls.Basic):
@@ -2677,6 +2693,9 @@ class NullEvaluatingTest(fixtures.MappedTest, testing.AssertsExecutionResults):
        class AltNameThing(cls.Basic):
            pass

+        class JSONThing(cls.Basic):
+            pass
+
    @classmethod
    def setup_mappers(cls):
        Thing = cls.classes.Thing
@@ -2686,6 +2705,9 @@ class NullEvaluatingTest(fixtures.MappedTest, testing.AssertsExecutionResults):

        mapper(AltNameThing, cls.tables.test_w_renames, column_prefix="_foo_")

+        if testing.requires.json_type.enabled:
+            mapper(cls.classes.JSONThing, cls.tables.test_has_json)
+
    def _assert_col(self, name, value):
        Thing, AltNameThing = self.classes.Thing, self.classes.AltNameThing
        s = Session()
@@ -2819,3 +2841,14 @@ class NullEvaluatingTest(fixtures.MappedTest, testing.AssertsExecutionResults):
        self._test_bulk_insert_novalue(
            "builtin_evals_null_default", "default_val"
        )
+
+    @testing.requires.json_type
+    def test_json_none_as_null(self):
+        JSONThing = self.classes.JSONThing
+
+        s = Session()
+        f1 = JSONThing(data=None, data_null=None)
+        s.add(f1)
+        s.commit()
+        eq_(s.query(cast(JSONThing.data, String)).scalar(), "null")
+        eq_(s.query(cast(JSONThing.data_null, String)).scalar(), None)
@@ -349,6 +349,10 @@ class DefaultRequirements(SuiteRequirements):
            "postgresql", "doesn't support sequences as a server side default."
        )

+    @property
+    def sql_expressions_inserted_as_primary_key(self):
+        return only_if([self.returning, self.sqlite])
+
    @property
    def correlated_outer_joins(self):
        """Target must support an outer join to a subquery which
@@ -4,12 +4,14 @@ from sqlalchemy.exc import CompileError
 from sqlalchemy.sql import and_
 from sqlalchemy.sql import bindparam
 from sqlalchemy.sql import column
+from sqlalchemy.sql import cte
 from sqlalchemy.sql import exists
 from sqlalchemy.sql import func
 from sqlalchemy.sql import literal
 from sqlalchemy.sql import select
 from sqlalchemy.sql import table
 from sqlalchemy.sql.elements import quoted_name
+from sqlalchemy.sql.selectable import CTE
 from sqlalchemy.sql.visitors import cloned_traverse
 from sqlalchemy.testing import assert_raises_message
 from sqlalchemy.testing import AssertsCompiledSQL
@@ -1126,3 +1128,28 @@ class CTETest(fixtures.TestBase, AssertsCompiledSQL):
            "UPDATE products SET id=:id, price=:price FROM pd "
            "WHERE products.price = pd.price",
        )
+
+    def test_standalone_function(self):
+        a = table("a", column("x"))
+        a_stmt = select([a])
+
+        stmt = select([cte(a_stmt)])
+
+        self.assert_compile(
+            stmt,
+            "WITH anon_1 AS (SELECT a.x AS x FROM a) "
+            "SELECT anon_1.x FROM anon_1",
+        )
+
+    def test_no_alias_construct(self):
+        a = table("a", column("x"))
+        a_stmt = select([a])
+
+        assert_raises_message(
+            NotImplementedError,
+            "The CTE class is not intended to be constructed directly.  "
+            r"Please use the cte\(\) standalone function",
+            CTE,
+            a_stmt,
+            "foo",
+        )
@@ -1088,6 +1088,90 @@ class ClauseAdapterTest(fixtures.TestBase, AssertsCompiledSQL):
            "AS anon_1 FROM table1 AS t1alias",
        )

+    def test_correlate_except_on_clone(self):
+        # test [ticket:4537]'s issue
+
+        t1alias = t1.alias("t1alias")
+        j = t1.join(t1alias, t1.c.col1 == t1alias.c.col2)
+
+        vis = sql_util.ClauseAdapter(j)
+
+        # "control" subquery - uses correlate which has worked w/ adaption
+        # for a long time
+        control_s = (
+            select([t2.c.col1])
+            .where(t2.c.col1 == t1.c.col1)
+            .correlate(t2)
+            .as_scalar()
+        )
+
+        # test subquery - given only t1 and t2 in the enclosing selectable,
+        # will do the same thing as the "control" query since the correlation
+        # works out the same
+        s = (
+            select([t2.c.col1])
+            .where(t2.c.col1 == t1.c.col1)
+            .correlate_except(t1)
+            .as_scalar()
+        )
+
+        # use both subqueries in statements
+        control_stmt = select([control_s, t1.c.col1, t2.c.col1]).select_from(
+            t1.join(t2, t1.c.col1 == t2.c.col1)
+        )
+
+        stmt = select([s, t1.c.col1, t2.c.col1]).select_from(
+            t1.join(t2, t1.c.col1 == t2.c.col1)
+        )
+        # they are the same
+        self.assert_compile(
+            control_stmt,
+            "SELECT "
+            "(SELECT table2.col1 FROM table1 "
+            "WHERE table2.col1 = table1.col1) AS anon_1, "
+            "table1.col1, table2.col1 "
+            "FROM table1 "
+            "JOIN table2 ON table1.col1 = table2.col1",
+        )
+        self.assert_compile(
+            stmt,
+            "SELECT "
+            "(SELECT table2.col1 FROM table1 "
+            "WHERE table2.col1 = table1.col1) AS anon_1, "
+            "table1.col1, table2.col1 "
+            "FROM table1 "
+            "JOIN table2 ON table1.col1 = table2.col1",
+        )
+
+        # now test against the adaption of "t1" into "t1 JOIN t1alias".
+        # note in the control case, we aren't actually testing that
+        # Select is processing the "correlate" list during the adaption
+        # since we aren't adapting the "correlate"
+        self.assert_compile(
+            vis.traverse(control_stmt),
+            "SELECT "
+            "(SELECT table2.col1 FROM "
+            "table1 JOIN table1 AS t1alias ON table1.col1 = t1alias.col2 "
+            "WHERE table2.col1 = table1.col1) AS anon_1, "
+            "table1.col1, table2.col1 "
+            "FROM table1 JOIN table1 AS t1alias ON table1.col1 = t1alias.col2 "
+            "JOIN table2 ON table1.col1 = table2.col1",
+        )
+
+        # but here, correlate_except() does have the thing we're adapting
+        # so whatever is in there has to be expanded out to include
+        # the adaptation target, in this case "t1 JOIN t1alias".
+        self.assert_compile(
+            vis.traverse(stmt),
+            "SELECT "
+            "(SELECT table2.col1 FROM "
+            "table1 JOIN table1 AS t1alias ON table1.col1 = t1alias.col2 "
+            "WHERE table2.col1 = table1.col1) AS anon_1, "
+            "table1.col1, table2.col1 "
+            "FROM table1 JOIN table1 AS t1alias ON table1.col1 = t1alias.col2 "
+            "JOIN table2 ON table1.col1 = table2.col1",
+        )
+
    @testing.fails_on_everything_except()
    def test_joins_dont_adapt(self):
        # adapting to a join, i.e. ClauseAdapter(t1.join(t2)), doesn't
@@ -16,6 +16,7 @@ from sqlalchemy import table
 from sqlalchemy import text
 from sqlalchemy.dialects import mysql
 from sqlalchemy.dialects import postgresql
+from sqlalchemy.dialects import sqlite
 from sqlalchemy.engine import default
 from sqlalchemy.sql import crud
 from sqlalchemy.testing import assert_raises
@@ -1196,6 +1197,91 @@ class MultirowTest(_InsertTestBase, fixtures.TablesTest, AssertsCompiledSQL):
            dialect=postgresql.dialect(),
        )

+    def test_sql_expression_pk_autoinc_lastinserted(self):
+        # test that postfetch isn't invoked for a SQL expression
+        # in a primary key column.  the DB either needs to support a lastrowid
+        # that can return it, or RETURNING.  [ticket:3133]
+        metadata = MetaData()
+        table = Table(
+            "sometable",
+            metadata,
+            Column("id", Integer, primary_key=True),
+            Column("data", String),
+        )
+
+        stmt = table.insert().return_defaults().values(id=func.foobar())
+        compiled = stmt.compile(dialect=sqlite.dialect(), column_keys=["data"])
+        eq_(compiled.postfetch, [])
+        eq_(compiled.returning, [])
+
+        self.assert_compile(
+            stmt,
+            "INSERT INTO sometable (id, data) VALUES " "(foobar(), ?)",
+            checkparams={"data": "foo"},
+            params={"data": "foo"},
+            dialect=sqlite.dialect(),
+        )
+
+    def test_sql_expression_pk_autoinc_returning(self):
+        # test that return_defaults() works with a primary key where we are
+        # sending a SQL expression, and we need to get the server-calculated
+        # value back.  [ticket:3133]
+        metadata = MetaData()
+        table = Table(
+            "sometable",
+            metadata,
+            Column("id", Integer, primary_key=True),
+            Column("data", String),
+        )
+
+        stmt = table.insert().return_defaults().values(id=func.foobar())
+        returning_dialect = postgresql.dialect()
+        returning_dialect.implicit_returning = True
+        compiled = stmt.compile(
+            dialect=returning_dialect, column_keys=["data"]
+        )
+        eq_(compiled.postfetch, [])
+        eq_(compiled.returning, [table.c.id])
+
+        self.assert_compile(
+            stmt,
+            "INSERT INTO sometable (id, data) VALUES "
+            "(foobar(), %(data)s) RETURNING sometable.id",
+            checkparams={"data": "foo"},
+            params={"data": "foo"},
+            dialect=returning_dialect,
+        )
+
+    def test_sql_expression_pk_noautoinc_returning(self):
+        # test that return_defaults() works with a primary key where we are
+        # sending a SQL expression, and we need to get the server-calculated
+        # value back.  [ticket:3133]
+        metadata = MetaData()
+        table = Table(
+            "sometable",
+            metadata,
+            Column("id", Integer, autoincrement=False, primary_key=True),
+            Column("data", String),
+        )
+
+        stmt = table.insert().return_defaults().values(id=func.foobar())
+        returning_dialect = postgresql.dialect()
+        returning_dialect.implicit_returning = True
+        compiled = stmt.compile(
+            dialect=returning_dialect, column_keys=["data"]
+        )
+        eq_(compiled.postfetch, [])
+        eq_(compiled.returning, [table.c.id])
+
+        self.assert_compile(
+            stmt,
+            "INSERT INTO sometable (id, data) VALUES "
+            "(foobar(), %(data)s) RETURNING sometable.id",
+            checkparams={"data": "foo"},
+            params={"data": "foo"},
+            dialect=returning_dialect,
+        )
+
    def test_python_fn_default(self):
        metadata = MetaData()
        table = Table(
@@ -60,8 +60,8 @@ class InsertExecTest(fixtures.TablesTest):
            exc.StatementError,
            r"\(sqlalchemy.exc.InvalidRequestError\) A value is required for "
            "bind parameter 'user_name', in "
-            "parameter group 2 "
-            r"\[SQL: u?'INSERT INTO users",
+            "parameter group 2\n"
+            r"\[SQL: u?INSERT INTO users",
            users.insert().execute,
            {"user_id": 7, "user_name": "jack"},
            {"user_id": 8, "user_name": "ed"},
@@ -1,14 +1,18 @@
 from sqlalchemy import Column
+from sqlalchemy import column
 from sqlalchemy import ForeignKey
 from sqlalchemy import Integer
 from sqlalchemy import join
 from sqlalchemy import lateral
 from sqlalchemy import String
 from sqlalchemy import Table
+from sqlalchemy import table
 from sqlalchemy import true
 from sqlalchemy.engine import default
 from sqlalchemy.sql import func
 from sqlalchemy.sql import select
+from sqlalchemy.sql.selectable import Lateral
+from sqlalchemy.testing import assert_raises_message
 from sqlalchemy.testing import AssertsCompiledSQL
 from sqlalchemy.testing import fixtures

@@ -150,3 +154,15 @@ class LateralTest(fixtures.TablesTest, AssertsCompiledSQL):
            "LATERAL generate_series(:generate_series_1, "
            "bookcases.bookcase_shelves) AS anon_1 ON true",
        )
+
+    def test_no_alias_construct(self):
+        a = table("a", column("x"))
+
+        assert_raises_message(
+            NotImplementedError,
+            "The Lateral class is not intended to be constructed directly.  "
+            r"Please use the lateral\(\) standalone",
+            Lateral,
+            a,
+            "foo",
+        )
@@ -28,6 +28,7 @@ from sqlalchemy import type_coerce
 from sqlalchemy import TypeDecorator
 from sqlalchemy import union
 from sqlalchemy import util
+from sqlalchemy.sql import Alias
 from sqlalchemy.sql import column
 from sqlalchemy.sql import elements
 from sqlalchemy.sql import expression
@@ -871,6 +872,18 @@ class SelectableTest(
        Table("t1", MetaData(), c1)
        eq_(c1._label, "t1_c1")

+    def test_no_alias_construct(self):
+        a = table("a", column("x"))
+
+        assert_raises_message(
+            NotImplementedError,
+            "The Alias class is not intended to be constructed directly.  "
+            r"Please use the alias\(\) standalone function",
+            Alias,
+            a,
+            "foo",
+        )
+

 class RefreshForNewColTest(fixtures.TestBase):
    def test_join_uninit(self):
@@ -1,12 +1,16 @@
 from sqlalchemy import Column
+from sqlalchemy import column
 from sqlalchemy import Integer
 from sqlalchemy import String
 from sqlalchemy import Table
+from sqlalchemy import table
 from sqlalchemy import tablesample
 from sqlalchemy.engine import default
 from sqlalchemy.sql import func
 from sqlalchemy.sql import select
 from sqlalchemy.sql import text
+from sqlalchemy.sql.selectable import TableSample
+from sqlalchemy.testing import assert_raises_message
 from sqlalchemy.testing import AssertsCompiledSQL
 from sqlalchemy.testing import fixtures

@@ -59,3 +63,16 @@ class TableSampleTest(fixtures.TablesTest, AssertsCompiledSQL):
            "SELECT alias.people_id FROM "
            "people AS alias TABLESAMPLE system(1)",
        )
+
+    def test_no_alias_construct(self):
+        a = table("a", column("x"))
+
+        assert_raises_message(
+            NotImplementedError,
+            "The TableSample class is not intended to be constructed "
+            "directly.  "
+            r"Please use the tablesample\(\) standalone",
+            TableSample,
+            a,
+            "foo",
+        )
@@ -249,6 +249,8 @@ class AdaptTest(fixtures.TestBase):
                        or t1.__dict__[k] is None
                    )

+            eq_(t1.evaluates_none().should_evaluate_none, True)
+
    def test_python_type(self):
        eq_(types.Integer().python_type, int)
        eq_(types.Numeric().python_type, decimal.Decimal)
@@ -82,6 +82,7 @@ commands=

 # thanks to https://julien.danjou.info/the-best-flake8-extensions/
 [testenv:pep8]
+basepython = python3.7
 deps=
      flake8
      flake8-import-order
Author	SHA1	Message	Date
rayluandMike Bayer	8acbc2624f	Expand joins when calculating PostgreSQL "WITH FOR UPDATE OF" Modified the :paramref:`.Select.with_for_update.of` parameter so that if a join or other composed selectable is passed, the individual :class:`.Table` objects will be filtered from it, allowing one to pass a join() object to the parameter, as occurs normally when using joined table inheritance with the ORM. Pull request courtesy Raymond Lu. Fixes: #4550 Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #4551 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4551 Pull-request-sha: 452da77d154a4087d530456db1c9af207d65cef4 Change-Id: If4b7c231f7b71190d7245543959fb5c3351125a1	2019-03-21 10:25:29 -04:00
ericdeansanchez	8eaccf16ac	Changed urllib.quote_plus to urllib.parse.quote_plus (Python 3 versio… Fixes: #4542 Closes: #4546 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4546 Pull-request-sha: 95d905cdcc33f44ff21d13e411e4a17843e6b54a Change-Id: Ibaf0852822e281e79770763e7e3550270d26cf66	2019-03-18 10:51:53 -04:00
Antony GelbergandMike Bayer	e209df356a	Clarify that declarative string resolution does not imply imports are implicit Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #4545 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4545 Pull-request-sha: c25221838d726f316dc04102b6b2002cec6b3a6c Change-Id: Ic38621f7deb3cb25e250aab61c5b2b02d633b187	2019-03-16 17:42:59 -04:00
mike bayer	4aadcca993	Merge "Clone internals for Select._correlate_except collection as well as _correlate"	2019-03-11 14:49:23 +00:00
Mike Bayer	25fc9562f0	Clone internals for Select._correlate_except collection as well as _correlate Fixed bug where use of :func:`.with_polymorphic` or other aliased construct would not properly adapt when the aliased target were used as the :meth:`.Select.correlate_except` target of a subquery used inside of a :func:`.column_property`. This required a fix to the clause adaption mechanics to properly handle a selectable that shows up in the "correlate except" list, in a similar manner as which occurs for selectables that show up in the "correlate" list. This is ultimately a fairly fundamental bug that has lasted for a long time but it is hard to come across it. Fixes: #4537 Change-Id: Ibb97d4eea18b3c452aad519dd14919bfb84d422f	2019-03-10 23:44:08 -04:00
Mike Bayer	16411cd8d6	Version 1.3.2 placeholder	2019-03-09 14:38:14 -05:00
Mike Bayer	f7d2556cd3	- 1.3.1	2019-03-09 14:32:32 -05:00
Mike Bayer	cbc31716c2	Commit transaction after SNAPSHOT isolation change A commit() is emitted after an isolation level change to SNAPSHOT, as both pyodbc and pymssql open an implicit transaction which blocks subsequent SQL from being emitted in the current transaction. Fixes: #4536 Change-Id: If3ba70f495bce2a35a873a3a72d1b30406e678c8	2019-03-08 23:45:25 -05:00
Mike Bayer	d514c032cd	black reformatting Change-Id: Idc48f1c80867cd953f9d611719d1d5b5d244ae38	2019-03-08 23:45:02 -05:00
Mike Bayer	7b0dfe3c46	Remove convert_unicode nodes and add version for deprecation Fixes: #4535 Change-Id: Ic5b1991ecef3b258573e1c59f5106a534f92a813	2019-03-08 14:42:24 -05:00
Mike Bayer	312d529dbb	Update the mysql-connector-python docs for now mysql-connector-python still seems to have issues with binary datatypes but the other issues here, with the possible exception of the deadlock still being investigated, seem to have been resolved in the recent 8.x series of mysql-connector-python. The driver is still not recommended due to these issues however we don't need a special class of warnings at this point. Change-Id: I2d59e76df5e84b34e318a24b468626900c39da46 References: #4531	2019-03-08 09:54:58 -05:00
Mike Bayer	e6d0963040	Add documentation section for cascade_scalar_deletes Change-Id: I56825652e0608862472bc594fc6c2b12ed5cc16f References: #4534	2019-03-07 09:54:45 -05:00
mike bayer	2160b7e652	Merge "Ensure association proxy works over synonym"	2019-03-06 16:38:49 +00:00
mike bayer	db212bcf49	Merge "Fix large_resultsets performance example field name"	2019-03-06 16:11:36 +00:00
mike bayer	0137b75136	Merge "Ensure scale param not sent to float types"	2019-03-06 15:47:01 +00:00
Matt Schuchhardt	203de6946e	Fix large_resultsets performance example field name Fixed bug in large_resultsets example case where a re-named "id" variable due to code reformatting caused the test to fail. Pull request courtesy Matt Schuchhardt. Fixes: #4528 Closes: #4529 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4529 Pull-request-sha: 8f52f10422114c397254db9e6f940d042708e0bd Change-Id: Iaca6fd06ec0ede8ff1745d627708ba492a2114c8	2019-03-06 10:34:17 -05:00
Mike Bayer	b78f460465	Ensure association proxy works over synonym Fixed regression where an association proxy linked to a synonym would no longer work, both at instance level and at class level. Fixes: #4522 Change-Id: I949079229ef87c12736c362df35444f6e30c8038	2019-03-06 10:16:49 -05:00
Mike Bayer	cb968b6dcd	Loosen SingletonThreadPoolTest This test is non-critical as the SingletonThreadPool is not production-level robust under multithreaded scenarios in any case. Fixes: #4527 Change-Id: Ie19ebd69438c97b2d8adb571f8f1b2c56894f7fb	2019-03-06 09:09:38 -05:00
Mike Bayer	2a2011389a	Ensure scale param not sent to float types Fixed regression in SQL Server reflection due to 🎫`4393` where the removal of open-ended ``**kw`` from the :class:`.Float` datatype caused reflection of this type to fail due to a "scale" argument being passed. Fixes: #4525 Change-Id: Ief8bb535778055eff2ab0d71660f81e3676390a1	2019-03-06 09:06:04 -05:00
Mike Bayer	201c4a60e4	run a black run fix some pep8s that snuck in Change-Id: Ied282007df30a52d232b1ba88659f2a123ff380f	2019-03-05 20:29:05 -05:00
Mike Bayer	66373c5811	Fix pooling typo eng -> engine Fixes: #4526 Change-Id: If117099b8521e15887ae916a5dbfcfa0b7cd594b	2019-03-05 19:42:41 -05:00
Mike Bayer	62bc03c51b	Merge remote-tracking branch 'origin/pr/4520' Change-Id: I3ca8e33c361ea088323e83437d6817ba02737de7	2019-03-05 12:39:07 -05:00
Mike Bayer	5c4b556c7a	Add a note that the truncation logic also raises an exception for too-long names. Change-Id: I8291a56235171827f5c41361b74118ba9c28c4c5	2019-03-05 12:13:08 -05:00
Mike Bayer	837d964004	- repair param inject to work around code examples to some degree - link to correct text() construct for CheckConstraint Change-Id: Ic46d1f79670a532f33bd9feb7f568dc3d1d6e38a	2019-03-05 11:35:15 -05:00
Mike Bayer	cfaf830a6f	Version 1.3.1 placeholder	2019-03-04 16:50:17 -05:00
Daniel Demmel	4f62c23cf3	Add missing attribute in hybrid.py docs	2019-03-04 18:10:22 +00:00
Mike Bayer	740bb50c22	- 1.3.0	2019-03-04 13:05:44 -05:00
mike bayer	8eda38e2aa	Merge "Use basepython 3.7 for flake8 tests"	2019-03-04 17:40:31 +00:00
Mike Bayer	b9d9d41691	Use basepython 3.7 for flake8 tests this to prevent flake8 from running with python 2 as we have py3k-specific checks Change-Id: I456c4b64c5bc04a81cd43de7dac1e803346f1245	2019-03-04 12:22:07 -05:00
mike bayer	f24659b72b	Merge "Add port comparison in __eq__() and __ne__() method to URL"	2019-03-04 15:45:53 +00:00
Sanjana	9268c320bf	Add port comparison in __eq__() and __ne__() method to URL Comparing two objects of :class:`.URL` using ``__eq__()`` did not take port number into consideration, two objects differing only by port number were considered equal. Port comparison is now added in ``__eq__()`` method of :class:`.URL`, objects differing by port number are now not equal. Additionally, ``__ne__()`` was not implemented for :class:`.URL` which caused unexpected result when ``!=`` was used in Python2, since there are no implied relationships among the comparison operators in Python2. Fixes: #4406 Closes: #4515 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4515 Pull-request-sha: 0f15b805f07e7fca1f82ca6c3aad98d50ea705b8 Change-Id: Iba7d224f1282dc3f4b884d1a746f2d46669f551e	2019-02-28 14:34:52 -05:00
Mike Bayer	a690ec0824	Add resolve_fks=False option for reflection Added new parameters :paramref:`.Table.resolve_fks` and :paramref:`.MetaData.reflect.resolve_fks` which when set to False will disable the automatic reflection of related tables encountered in :class:`.ForeignKey` objects, which can both reduce SQL overhead for omitted tables as well as avoid tables that can't be reflected for database-specific reasons. Two :class:`.Table` objects present in the same :class:`.MetaData` collection can still refer to each other even if the reflection of the two tables occurred separately. Fixes: #4517 Change-Id: I623baed42042a16c5109e4c8af6b2f64d2d00f95	2019-02-28 13:49:09 -05:00
Aidan Feldman	358a0dcf10	link to PyHive as a dialect	2019-02-27 16:01:04 -05:00
Ville Brofeldt	dc48ac5489	Ignore Pipenv* and .idea	2019-02-23 18:02:42 +02:00
mike bayer	53c7fc4690	Merge "Add test to as_interface when no cls or methods"	2019-02-22 00:33:55 +00:00
mike bayer	0cdc89c276	Merge "Prevent __init__ from being called for Alias, subclasses"	2019-02-22 00:33:31 +00:00
Toni Karppi	40401d7933	Add test to as_interface when no cls or methods Fixes #4511 ### Description The `as_interface` function requires that an argument be provided for at least one of the parameters `cls` or `methods`. There was no test that tested the case when neither of these arguments were provided. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. Have a nice day! Closes: #4512 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4512 Pull-request-sha: 156f6af95ede730d2e1a9d2c935c830cf3cfc96e Change-Id: Ifa23dbe1dbe9a824fe680e4947b81c96a50c133f	2019-02-21 14:22:48 -05:00
mike bayer	eed102f465	Merge "Add support for key-word based get()"	2019-02-21 19:19:49 +00:00
Mike Bayer	4ca3092c0a	Prevent __init__ from being called for Alias, subclasses The :class:`.Alias` class and related subclasses :class:`.CTE`, :class:`.Lateral` and :class:`.TableSample` have been reworked so that it is not possible for a user to construct the objects directly. These constructs require that the standalone construction function or selectable-bound method be used to instantiate new objects. Fixes: #4509 Change-Id: I74ae4786cb3ae625dab33b00bfd6bdc4e1219139	2019-02-21 11:18:19 -05:00
mike bayer	0a1c640531	Merge "Include newlines in StatementError formatting"	2019-02-21 15:52:29 +00:00
Nate Clark	8f318692d4	Include newlines in StatementError formatting Revised the formatting for :class:`.StatementError` when stringified. Each error detail is broken up over multiple newlines instead of spaced out on a single line. Additionally, the SQL representation now stringifies the SQL statement rather than using ``repr()``, so that newlines are rendered as is. Pull request courtesy Nate Clark. Fixes: #4500 Closes: #4501 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4501 Pull-request-sha: 60cc0ee68dc96b8f483a60d37bcb26b6c6d53efe Change-Id: I79d8418b7495e5691c9a56f41e79495c26a967ff	2019-02-20 18:56:47 -05:00
sanjana	c89a93e953	Add support for key-word based get() The :meth:`.Query.get` method can now accept a dictionary of attribute keys and values as a means of indicating the primary key value to load; is particularly useful for composite primary keys. Pull request courtesy Sanjana S. Fixes: #4316 Closes: #4505 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4505 Pull-request-sha: cfa8297ad2490be9eae24ec8b1a691e43cd75868 Change-Id: Ib19e7d51599a36f4878119c2f801c5c694793422	2019-02-20 17:08:34 -05:00
Mike Bayer	7a4c40ff6b	Remove inadvertent test_schema.db commit also revise .gitignore Change-Id: I93d900287fe35441db827203b9fdf63eb681ca69	2019-02-20 13:06:20 -05:00
mike bayer	d879bed878	Merge "Add NCHAR to Oracle dialect"	2019-02-20 17:54:07 +00:00
Mike Bayer	a3135c494e	revise timestamp correction scheme For very slow CI this test still fails so establish the date as being between the range of what the operation took Change-Id: I9204cd1915777ebc58bd598065b9015befb93345	2019-02-20 12:52:40 -05:00
Mike Bayer	d71f34cb2a	Ensure _simple_lazy_clause bind names are fixed before cloning Fixed a regression in 1.2 due to the introduction of baked queries for relationship lazy loaders, where a race condition is created during the generation of the "lazy clause" which occurs within a memoized attribute. If two threads initialize the memoized attribute concurrently, the baked query could be generated with bind parameter keys that are then replaced with new keys by the next run, leading to a lazy load query that specifies the related criteria as ``None``. The fix establishes that the parameter names are fixed before the new clause and parameter objects are generated, so that the names are the same every time. Fixes: #4507 Change-Id: I605b824e028c87bc20ca8c2577227cdf6a591064	2019-02-19 19:46:17 -05:00
Mike Bayer	5fe9da55c7	Add NCHAR to Oracle dialect Added support for reflection of the :class:`.NCHAR` datatype to the Oracle dialect, and added :class:`.NCHAR` to the list of types exported by the Oracle dialect. Fixes: #4506 Change-Id: I3a120b9ad714cbd0ebd8064519f43cf57e99d920	2019-02-19 12:13:04 -05:00
Mike Bayer	c0bd36873b	cherry-pick changelog update for 1.2.19	2019-02-15 13:08:58 -05:00
Mike Bayer	11f5ea1f7f	cherry-pick changelog from 1.2.18	2019-02-15 13:08:57 -05:00
Mike Bayer	c1b87bc95d	- formatting fix Change-Id: I93126865ffb5cc6486464402ed7ef17027441886	2019-02-15 12:38:49 -05:00
mike bayer	4d7ac4be6c	Merge "Adding setter to should_evaluate_none property"	2019-02-14 15:40:06 +00:00
sanjana	12dad561a7	Adding setter to should_evaluate_none property Fixed issue where the :class:`.JSON` type had a read-only :attr:`.JSON.should_evaluate_none` attribute, which would cause failures when making use of the :meth:`.TypeEngine.evaluates_none` method in conjunction with this type. Pull request courtesy Sanjana S. Fixes: #4485 Closes: #4496 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4496 Pull-request-sha: 044beb23982d411be6fe640716b1b693df0f7189 Change-Id: I1f3e1d7dec9d2ceb6ccaaa8cac158a062cf02710	2019-02-13 17:29:13 -05:00
Mike Bayer	71d642711d	Set IDENTITY_INSERT for insert.values({column: expr}) Fixed bug where the SQL Server "IDENTITY_INSERT" logic that allows an INSERT to proceed with an explicit value on an IDENTITY column was not detecting the case where :meth:`.Insert.values` were used with a dictionary that contained a :class:`.Column` as key and a SQL expression as a value. Fixes: #4499 Change-Id: Ia61cd6524b030b40a665db9c20771f0c5aa5fcd7	2019-02-13 11:26:54 -05:00
mike bayer	5cc5234e0b	Merge "Allow SQL expression for ORM primary keys"	2019-02-13 01:35:15 +00:00
Mike Bayer	c1f310df44	Allow SQL expression for ORM primary keys A SQL expression can now be assigned to a primary key attribute for an ORM flush in the same manner as ordinary attributes as described in :ref:`flush_embedded_sql_expressions` where the expression will be evaulated and then returned to the ORM using RETURNING, or in the case of pysqlite, works using the cursor.lastrowid attribute.Requires either a database that supports RETURNING (e.g. Postgresql, Oracle, SQL Server) or pysqlite. Fixes: #3133 Fixes: #4494 Change-Id: I83da8357354de002cb04fa4a553f2a2f90c5157d	2019-02-12 16:55:48 -05:00
mike bayer	104625941d	Merge "Add complete coverage and fix lower() for MySQL 88718 workaround"	2019-02-11 20:22:27 +00:00
Mike Bayer	49197c7b36	Add complete coverage and fix lower() for MySQL 88718 workaround Fixed a second regression caused by 🎫`4344` (the first was 🎫`4361`), which works around MySQL issue 88718, where the lower casing function used was not correct for Python 2 with OSX/Windows casing conventions, which would then raise ``TypeError``. Full coverage has been added to this logic so that every codepath is exercised in a mock style for all three casing conventions on all versions of Python. MySQL 8.0 has meanwhile fixed issue 88718 so the workaround is only applies to a particular span of MySQL 8.0 versions. Fixes: #4492 Change-Id: I14e7237e0be4a9c21c58c921066304ae99ac4dc6	2019-02-11 09:11:37 -05:00
mike bayer	bb7b353d6f	Merge "Don't run pending_to_persistent for non-new objects"	2019-02-10 03:57:10 +00:00
Mike Bayer	b2afef966d	Don't run pending_to_persistent for non-new objects Fixed fairly simple but critical issue where the :meth:`.SessionEvents.pending_to_persistent` event would be invoked for objects not just when they move from pending to persistent, but when they were also already persistent and just being updated, thus causing the event to be invoked for all objects on every update. Fixes: #4489 Change-Id: Ibe147020aa62f7d605cb1029b7f3b776f42e6b43	2019-02-09 14:38:43 -05:00
Mike Bayer	833583458c	Note for history methods that history is reset per-flush. Change-Id: I9bc4d0ddfa93f13e6717b89fa9934f1b8052147f	2019-02-09 12:22:39 -05:00
Mike Bayer	fcb5dbf395	mention no_autoflush in the flush section of the docs also remove link to wiki which is obsolete Change-Id: I9ebd578923c3eba6e35f3b6203c01ba003dd4115	2019-02-08 17:48:51 -05:00
Mike Bayer	6c177df67c	Version 1.3.0 placeholder	2019-02-08 15:19:40 -05:00