[orm] [bug] Fixed issue involving lazy="raise" strategy where an ORM delete of an
object would raise for a simple “use-get” style many-to-one relationship
that had lazy=”raise” configured. This is inconsistent vs. the change
introduced in 1.3 as part of #4353, where it was established that
a history operation that does not expect emit SQL should bypass the
lazy="raise" check, and instead effectively treat it as
lazy="raise_on_sql" for this case. The fix adjusts the lazy loader
strategy to not raise for the case where the lazy load was instructed that
it should not emit SQL if the object were not present.¶
References: #4997
[orm] [bug] Fixed regression introduced in 1.3.0 related to the association proxy
refactor in #4351 that prevented composite() attributes
from working in terms of an association proxy that references them.¶
References: #5000
[orm] [bug] Setting persistence-related flags on relationship() while also
setting viewonly=True will now emit a regular warning, as these flags do
not make sense for a viewonly=True relationship. In particular, the
“cascade” settings have their own warning that is generated based on the
individual values, such as “delete, delete-orphan”, that should not apply
to a viewonly relationship. Note however that in the case of “cascade”,
these settings are still erroneously taking effect even though the
relationship is set up as “viewonly”. In 1.4, all persistence-related
cascade settings will be disallowed on a viewonly=True relationship in
order to resolve this issue.¶
References: #4993
[orm] [bug] [py3k] Fixed issue where when assigning a collection to itself as a slice, the
mutation operation would fail as it would first erase the assigned
collection inadvertently. As an assignment that does not change the
contents should not generate events, the operation is now a no-op. Note
that the fix only applies to Python 3; in Python 2, the __setitem__
hook isn’t called in this case; __setslice__ is used instead which
recreates the list item-by-item in all cases.¶
References: #4990
[orm] [bug] Fixed issue where by if the “begin” of a transaction failed at the Core
engine/connection level, such as due to network error or database is locked
for some transactional recipes, within the context of the Session
procuring that connection from the conneciton pool and then immediately
returning it, the ORM Session would not close the connection
despite this connection not being stored within the state of that
Session. This would lead to the connection being cleaned out by
the connection pool weakref handler within garbage collection which is an
unpreferred codepath that in some special configurations can emit errors in
standard error.¶
References: #5034
[sql] [bug] Fixed bug where “distinct” keyword passed to select() would not
treat a string value as a “label reference” in the same way that the
select.distinct() does; it would instead raise unconditionally. This
keyword argument and the others passed to select() will ultimately
be deprecated for SQLAlchemy 2.0.¶
References: #5028
[sql] [bug] Changed the text of the exception for “Can’t resolve label reference” to include other kinds of label coercions, namely that “DISTINCT” is also in this category under the PostgreSQL dialect.¶
[sqlite] [bug] Fixed issue to workaround SQLite’s behavior of assigning “numeric” affinity to JSON datatypes, first described at Support for SQLite JSON Added, which returns scalar numeric JSON values as a number and not as a string that can be JSON deserialized. The SQLite-specific JSON deserializer now gracefully degrades for this case as an exception and bypasses deserialization for single numeric values, as from a JSON perspective they are already deserialized.¶
References: #5014
[mssql] [bug] Repaired support for the mssql.DATETIMEOFFSET datatype on PyODBC,
by adding PyODBC-level result handlers as it does not include native
support for this datatype. This includes usage of the Python 3 “timezone”
tzinfo subclass in order to set up a timezone, which on Python 2 makes
use of a minimal backport of “timezone” in sqlalchemy.util.¶
References: #4983
[orm] [bug] The relationship.omit_join flag was not intended to be
manually set to True, and will now emit a warning when this occurs. The
omit_join optimization is detected automatically, and the omit_join
flag was only intended to disable the optimization in the hypothetical case
that the optimization may have interfered with correct results, which has
not been observed with the modern version of this feature. Setting the
flag to True when it is not automatically detected may cause the selectin
load feature to not work correctly when a non-default primary join
condition is in use.¶
References: #4954
[orm] [bug] A warning is emitted if a primary key value is passed to Query.get()
that consists of None for all primary key column positions. Previously,
passing a single None outside of a tuple would raise a TypeError and
passing a composite None (tuple of None values) would silently pass
through. The fix now coerces the single None into a tuple where it is
handled consistently with the other None conditions. Thanks to Lev
Izraelit for the help with this.¶
References: #4915
[orm] [bug] The BakedQuery will not cache a query that was modified by a
QueryEvents.before_compile() event, so that compilation hooks that
may be applying ad-hoc modifications to queries will take effect on each
run. In particular this is helpful for events that modify queries used in
lazy loading as well as eager loading such as “select in” loading. In
order to re-enable caching for a query modified by this event, a new
flag bake_ok is added; see Using the before_compile event for
details.
A longer term plan to provide a new form of SQL caching should solve this kind of issue more comprehensively.
¶References: #4947
[orm] [bug] Fixed ORM bug where a “secondary” table that referred to a selectable which
in some way would refer to the local primary table would apply aliasing to
both sides of the join condition when a relationship-related join, either
via Query.join() or by joinedload(), were generated. The
“local” side is now excluded.¶
References: #4974
[orm] [usecase] Added accessor Query.is_single_entity() to Query, which
will indicate if the results returned by this Query will be a
list of ORM entities, or a tuple of entities or column expressions.
SQLAlchemy hopes to improve upon the behavior of single entity / tuples in
future releases such that the behavior would be explicit up front, however
this attribute should be helpful with the current behavior. Pull request
courtesy Patrick Hayes.¶
References: #4934
[engine] [bug] Fixed bug where parameter repr as used in logging and error reporting needs additional context in order to distinguish between a list of parameters for a single statement and a list of parameter lists, as the “list of lists” structure could also indicate a single parameter list where the first parameter itself is a list, such as for an array parameter. The engine/connection now passes in an additional boolean indicating how the parameters should be considered. The only SQLAlchemy backend that expects arrays as parameters is that of psycopg2 which uses pyformat parameters, so this issue has not been too apparent, however as other drivers that use positional gain more features it is important that this be supported. It also eliminates the need for the parameter repr function to guess based on the parameter structure passed.¶
References: #4902
[engine] [bug] [postgresql] Fixed bug in Inspector where the cache key generation did not
take into account arguments passed in the form of tuples, such as the tuple
of view name styles to return for the PostgreSQL dialect. This would lead
the inspector to cache too generally for a more specific set of criteria.
The logic has been adjusted to include every keyword element in the cache,
as every argument is expected to be appropriate for a cache else the
caching decorator should be bypassed by the dialect.¶
References: #4955
[sql] [bug] [py3k] Changed the repr() of the quoted_name construct to use
regular string repr() under Python 3, rather than running it through
“backslashreplace” escaping, which can be misleading.¶
References: #4931
[sql] [usecase] Added new accessors to expressions of type JSON to allow for
specific datatype access and comparison, covering strings, integers,
numeric, boolean elements. This revises the documented approach of
CASTing to string when comparing values, instead adding specific
functionality into the PostgreSQL, SQlite, MySQL dialects to reliably
deliver these basic types in all cases.
See also
References: #4276
[sql] [usecase] The text() construct now supports “unique” bound parameters, which
will dynamically uniquify themselves on compilation thus allowing multiple
text() constructs with the same bound parameter names to be combined
together.¶
References: #4933
[schema] [bug] Fixed bug where a table that would have a column label overlap with a plain
column name, such as “foo.id AS foo_id” vs. “foo.foo_id”, would prematurely
generate the ._label attribute for a column before this overlap could
be detected due to the use of the index=True or unique=True flag on
the column in conjunction with the default naming convention of
"column_0_label". This would then lead to failures when ._label
were used later to generate a bound parameter name, in particular those
used by the ORM when generating the WHERE clause for an UPDATE statement.
The issue has been fixed by using an alternate ._label accessor for DDL
generation that does not affect the state of the Column. The
accessor also bypasses the key-deduplication step as it is not necessary
for DDL, the naming is now consistently "<tablename>_<columnname>"
without any subsequent numeric symbols when used in DDL.¶
References: #4911
[schema] [usecase] Added DDL support for “computed columns”; these are DDL column specifications for columns that have a server-computed value, either upon SELECT (known as “virtual”) or at the point of which they are INSERTed or UPDATEd (known as “stored”). Support is established for Postgresql, MySQL, Oracle SQL Server and Firebird. Thanks to Federico Caselli for lots of work on this one.
¶References: #4894
[mssql] [bug] Fixed issue in MSSQL dialect where an expression-based OFFSET value in a SELECT would be rejected, even though the dialect can render this expression inside of a ROW NUMBER-oriented LIMIT/OFFSET construct.¶
References: #4973
[mssql] [bug] Fixed an issue in the Engine.table_names() method where it would
feed the dialect’s default schema name back into the dialect level table
function, which in the case of SQL Server would interpret it as a
dot-tokenized schema name as viewed by the mssql dialect, which would
cause the method to fail in the case where the database username actually
had a dot inside of it. In 1.3, this method is still used by the
MetaData.reflect() function so is a prominent codepath. In 1.4,
which is the current master development branch, this issue doesn’t exist,
both because MetaData.reflect() isn’t using this method nor does the
method pass the default schema name explicitly. The fix nonetheless
guards against the default server name value returned by the dialect from
being interpreted as dot-tokenized name under any circumstances by
wrapping it in quoted_name().¶
References: #4923
[oracle] [bug] [firebird] Modified the approach of “name normalization” for the Oracle and Firebird
dialects, which converts from the UPPERCASE-as-case-insensitive convention
of these dialects into lowercase-as-case-insensitive for SQLAlchemy, to not
automatically apply the quoted_name construct to a name that
matches itself under upper or lower case conversion, as is the case for
many non-european characters. All names used within metadata structures
are converted to quoted_name objects in any case; the change
here would only affect the output of some inspection functions.¶
References: #4931
[oracle] [bug] The sqltypes.NCHAR datatype will now bind to the
cx_Oracle.FIXED_NCHAR DBAPI data bindings when used in a bound
parameter, which supplies proper comparison behavior against a
variable-length string. Previously, the sqltypes.NCHAR datatype
would bind to cx_oracle.NCHAR which is not fixed length; the
sqltypes.CHAR datatype already binds to cx_Oracle.FIXED_CHAR
so it is now consistent that sqltypes.NCHAR binds to
cx_Oracle.FIXED_NCHAR.¶
References: #4913
[oracle] [usecase] Added dialect-level flag encoding_errors to the cx_Oracle dialect,
which can be specified as part of create_engine(). This is passed
to SQLAlchemy’s unicode decoding converter under Python 2, and to
cx_Oracle’s cursor.var() object as the encodingErrors parameter
under Python 3, for the very unusual case that broken encodings are present
in the target database which cannot be fetched unless error handling is
relaxed. The value is ultimately one of the Python “encoding errors”
parameters passed to decode().¶
References: #4799
[bug] [tests] Fixed test failures which would occur with newer SQLite as of version 3.30 or greater, due to their addition of nulls ordering syntax as well as new restrictions on aggregate functions. Pull request courtesy Nils Philippsen.¶
References: #4920
[bug] [installation] [windows] Added a workaround for a setuptools-related failure that has been observed as occurring on Windows installations, where setuptools is not correctly reporting a build error when the MSVC build dependencies are not installed and therefore not allowing graceful degradation into non C extensions builds.¶
References: #4967
[oracle] [bug] Fixed regression in Oracle dialect that was inadvertently using max identifier length of 128 characters on Oracle server 12.2 and greater even though the stated contract for the remainder of the 1.3 series is that this value stays at 30 until version SQLAlchemy 1.4. Also repaired issues with the retrieval of the “compatibility” version, and removed the warning emitted when the “v$parameter” view was not accessible as this was causing user confusion.¶
[orm] [bug] Fixed regression in selectinload loader strategy caused by #4775 (released in version 1.3.6) where a many-to-one attribute of None would no longer be populated by the loader. While this was usually not noticeable due to the lazyloader populating None upon get, it would lead to a detached instance error if the object were detached.¶
References: #4872
[orm] [bug] Passing a plain string expression to Session.query() is deprecated,
as all string coercions were removed in #4481 and this one should
have been included. The literal_column() function may be used to
produce a textual column expression.¶
References: #4873
[orm] [bug] A warning is emitted for a condition in which the Session may
implicitly swap an object out of the identity map for another one with the
same primary key, detaching the old one, which can be an observed result of
load operations which occur within the SessionEvents.after_flush()
hook. The warning is intended to notify the user that some special
condition has caused this to happen and that the previous object may not be
in the expected state.¶
References: #4890
[engine] [usecase] Added new create_engine() parameter
create_engine.max_identifier_length. This overrides the
dialect-coded “max identifier length” in order to accommodate for databases
that have recently changed this length and the SQLAlchemy dialect has
not yet been adjusted to detect for that version. This parameter interacts
with the existing create_engine.label_length parameter in that
it establishes the maximum (and default) value for anonymously generated
labels. Additionally, post-connection detection of max identifier lengths
has been added to the dialect system. This feature is first being used
by the Oracle dialect.
See also
Max Identifier Lengths - in the Oracle dialect documentation
References: #4857
[sql] [bug] Characters that interfere with “pyformat” or “named” formats in bound
parameters, namely %, (, ) and the space character, as well as a few
other typically undesirable characters, are stripped early for a
bindparam() that is using an anonymized name, which is typically
generated automatically from a named column which itself includes these
characters in its name and does not use a .key, so that they do not
interfere either with the SQLAlchemy compiler’s use of string formatting or
with the driver-level parsing of the parameter, both of which could be
demonstrated before the fix. The change only applies to anonymized
parameter names that are generated and consumed internally, not end-user
defined names, so the change should have no impact on any existing code.
Applies in particular to the psycopg2 driver which does not otherwise quote
special parameter names, but also strips leading underscores to suit Oracle
(but not yet leading numbers, as some anon parameters are currently
entirely numeric/underscore based); Oracle in any case continues to quote
parameter names that include special characters.¶
References: #4837
[sql] [usecase] Added an explicit error message for the case when objects passed to
Table are not SchemaItem objects, rather than resolving
to an attribute error.¶
References: #4847
[mssql] [bug] Added identifier quoting to the schema name applied to the “use” statement
which is invoked when a SQL Server multipart schema name is used within a
Table that is being reflected, as well as for Inspector
methods such as Inspector.get_table_names(); this accommodates for
special characters or spaces in the database name. Additionally, the “use”
statement is not emitted if the current database matches the target owner
database name being passed.¶
References: #4883
[oracle] [bug] Restored adding cx_Oracle.DATETIME to the setinputsizes() call when a
SQLAlchemy Date, DateTime or Time datatype is
used, as some complex queries require this to be present. This was removed
in the 1.2 series for arbitrary reasons.¶
References: #4886
[oracle] [usecase] The Oracle dialect now emits a warning if Oracle version 12.2 or greater is
used, and the create_engine.max_identifier_length parameter is
not set. The version in this specific case defaults to that of the
“compatibility” version set in the Oracle server configuration, not the
actual server version. In version 1.4, the default max_identifier_length
for 12.2 or greater will move to 128 characters. In order to maintain
forwards compatibility, applications should set
create_engine.max_identifier_length to 30 in order to maintain
the same length behavior, or to 128 in order to test the upcoming behavior.
This length determines among other things how generated constraint names
are truncated for statements like CREATE CONSTRAINT and DROP
CONSTRAINT, which means a the new length may produce a name-mismatch
against a name that was generated with the old length, impacting database
migrations.
See also
Max Identifier Lengths - in the Oracle dialect documentation
References: #4857
[bug] [tests] Fixed unit test regression released in 1.3.8 that would cause failure for Oracle, SQL Server and other non-native ENUM platforms due to new enumeration tests added as part of #4285 enum sortability in the unit of work; the enumerations created constraints that were duplicated on name.¶
References: #4285
[orm] [bug] Fixed bug where Load objects were not pickleable due to
mapper/relationship state in the internal context dictionary. These
objects are now converted to picklable using similar techniques as that of
other elements within the loader option system that have long been
serializable.¶
References: #4823
[orm] [usecase] Added support for the use of an Enum datatype using Python
pep-435 enumeration objects as values for use as a primary key column
mapped by the ORM. As these values are not inherently sortable, as
required by the ORM for primary keys, a new
TypeEngine.sort_key_function attribute is added to the typing
system which allows any SQL type to implement a sorting for Python objects
of its type which is consulted by the unit of work. The Enum
type then defines this using the database value of a given enumeration.
The sorting scheme can be also be redefined by passing a callable to the
Enum.sort_key_function parameter. Pull request courtesy
Nicolas Caniart.¶
References: #4285
[engine] [feature] Added new parameter create_engine.hide_parameters which when
set to True will cause SQL parameters to no longer be logged, nor rendered
in the string representation of a StatementError object.¶
References: #4815
[engine] [bug] Fixed an issue whereby if the dialect “initialize” process which occurs on
first connect would encounter an unexpected exception, the initialize
process would fail to complete and then no longer attempt on subsequent
connection attempts, leaving the dialect in an un-initialized, or partially
initialized state, within the scope of parameters that need to be
established based on inspection of a live connection. The “invoke once”
logic in the event system has been reworked to accommodate for this
occurrence using new, private API features that establish an “exec once”
hook that will continue to allow the initializer to fire off on subsequent
connections, until it completes without raising an exception. This does not
impact the behavior of the existing once=True flag within the event
system.¶
References: #4807
[postgresql] [bug] Revised the approach for the just added support for the psycopg2 “execute_values()” feature added in 1.3.7 for #4623. The approach relied upon a regular expression that would fail to match for a more complex INSERT statement such as one which had subqueries involved. The new approach matches exactly the string that was rendered as the VALUES clause.¶
References: #4623
[postgresql] [bug] Fixed bug where Postgresql operators such as
postgresql.ARRAY.Comparator.contains() and
postgresql.ARRAY.Comparator.contained_by() would fail to function
correctly for non-integer values when used against a
postgresql.array object, due to an erroneous assert statement.¶
References: #4822
[postgresql] [usecase] Added support for reflection of CHECK constraints that include the special
PostgreSQL qualifier “NOT VALID”, which can be present for CHECK
constraints that were added to an exsiting table with the directive that
they not be applied to existing data in the table. The PostgreSQL
dictionary for CHECK constraints as returned by
Inspector.get_check_constraints() may include an additional entry
dialect_options which within will contain an entry "not_valid":
True if this symbol is detected. Pull request courtesy Bill Finn.¶
References: #4824
[sqlite] [bug] [reflection] Fixed bug where a FOREIGN KEY that was set up to refer to the parent table by table name only without the column names would not correctly be reflected as far as setting up the “referred columns”, since SQLite’s PRAGMA does not report on these columns if they weren’t given explicitly. For some reason this was harcoded to assume the name of the local column, which might work for some cases but is not correct. The new approach reflects the primary key of the referred table and uses the constraint columns list as the referred columns list, if the remote column(s) aren’t present in the reflected pragma directly.¶
References: #4810
[orm] [bug] Fixed regression caused by new selectinload for many-to-one logic where a primaryjoin condition not based on real foreign keys would cause KeyError if a related object did not exist for a given key value on the parent object.¶
References: #4777
[orm] [bug] Fixed bug where using Query.first() or a slice expression in
conjunction with a query that has an expression based “offset” applied
would raise TypeError, due to an “or” conditional against “offset” that did
not expect it to be a SQL expression as opposed to an integer or None.¶
References: #4803
[sql] [bug] Fixed issue where Index object which contained a mixture of
functional expressions which were not resolvable to a particular column,
in combination with string-based column names, would fail to initialize
its internal state correctly leading to failures during DDL compilation.¶
References: #4778
[sql] [bug] Fixed bug where TypeEngine.column_expression() method would not be
applied to subsequent SELECT statements inside of a UNION or other
CompoundSelect, even though the SELECT statements are rendered at
the topmost level of the statement. New logic now differentiates between
rendering the column expression, which is needed for all SELECTs in the
list, vs. gathering the returned data type for the result row, which is
needed only for the first SELECT.¶
References: #4787
[sql] [bug] Fixed issue where internal cloning of SELECT constructs could lead to a key error if the copy of the SELECT changed its state such that its list of columns changed. This was observed to be occurring in some ORM scenarios which may be unique to 1.3 and above, so is partially a regression fix.¶
References: #4780
[postgresql] [usecase] Added new dialect flag for the psycopg2 dialect, executemany_mode which
supersedes the previous experimental use_batch_mode flag.
executemany_mode supports both the “execute batch” and “execute values”
functions provided by psycopg2, the latter which is used for compiled
insert() constructs. Pull request courtesy Yuval Dinari.
See also
References: #4623
[mysql] [bug] The MySQL dialects will emit “SET NAMES” at the start of a connection when charset is given to the MySQL driver, to appease an apparent behavior observed in MySQL 8.0 that raises a collation error when a UNION includes string columns unioned against columns of the form CAST(NULL AS CHAR(..)), which is what SQLAlchemy’s polymorphic_union function does. The issue seems to have affected PyMySQL for at least a year, however has recently appeared as of mysqlclient 1.4.4 based on changes in how this DBAPI creates a connection. As the presence of this directive impacts three separate MySQL charset settings which each have intricate effects based on their presense, SQLAlchemy will now emit the directive on new connections to ensure correct behavior.¶
References: #4804
[mysql] [bug] Added another fix for an upstream MySQL 8 issue where a case sensitive table name is reported incorrectly in foreign key constraint reflection, this is an extension of the fix first added for #4344 which affects a case sensitive column name. The new issue occurs through MySQL 8.0.17, so the general logic of the 88718 fix remains in place.
See also
https://bugs.mysql.com/bug.php?id=96365 - upstream bug
References: #4751
[mysql] [usecase] Added reserved words ARRAY and MEMBER to the MySQL reserved words list, as MySQL 8.0 has now made these reserved.¶
References: #4783
[sqlite] [bug] The dialects that support json are supposed to take arguments
json_serializer and json_deserializer at the create_engine() level,
however the SQLite dialect calls them _json_serilizer and
_json_deserilalizer. The names have been corrected, the old names are
accepted with a change warning, and these parameters are now documented as
create_engine.json_serializer and
create_engine.json_deserializer.¶
References: #4798
[sqlite] [bug] Fixed bug where usage of “PRAGMA table_info” in SQLite dialect meant that reflection features to detect for table existence, list of table columns, and list of foreign keys, would default to any table in any attached database, when no schema name was given and the table did not exist in the base schema. The fix explicitly runs PRAGMA for the ‘main’ schema and then the ‘temp’ schema if the ‘main’ returned no rows, to maintain the behavior of tables + temp tables in the “no schema” namespace, attached tables only in the “schema” namespace.¶
References: #4793
[mssql] [usecase] Added new mssql.try_cast() construct for SQL Server which emits
“TRY_CAST” syntax. Pull request courtesy Leonel Atencio.¶
References: #4782
[bug] [events] Fixed issue in event system where using the once=True flag with
dynamically generated listener functions would cause event registration of
future events to fail if those listener functions were garbage collected
after they were used, due to an assumption that a listened function is
strongly referenced. The “once” wrapped is now modified to strongly
reference the inner function persistently, and documentation is updated
that using “once” does not imply automatic de-registration of listener
functions.¶
References: #4794
[orm] [feature] Added new loader option method Load.options() which allows loader
options to be constructed hierarchically, so that many sub-options can be
applied to a particular path without needing to call defaultload()
many times. Thanks to Alessio Bogon for the idea.¶
References: #4736
[orm] [bug] Fixed regression caused by #4365 where a join from an entity to itself without using aliases no longer raises an informative error message, instead failing on an assertion. The informative error condition has been restored.¶
References: #4773
[orm] [bug] Fixed an issue where the orm._ORMJoin.join() method, which is a
not-internally-used ORM-level method that exposes what is normally an
internal process of Query.join(), did not propagate the full and
outerjoin keyword arguments correctly. Pull request courtesy Denis
Kataev.¶
References: #4713
[orm] [bug] Fixed bug where a many-to-one relationship that specified uselist=True
would fail to update correctly during a primary key change where a related
column needs to change.¶
References: #4772
[orm] [bug] Fixed bug where the detection for many-to-one or one-to-one use with a
“dynamic” relationship, which is an invalid configuration, would fail to
raise if the relationship were configured with uselist=True. The
current fix is that it warns, instead of raises, as this would otherwise be
backwards incompatible, however in a future release it will be a raise.¶
References: #4772
[orm] [bug] Fixed bug where a synonym created against a mapped attribute that does not exist yet, as is the case when it refers to backref before mappers are configured, would raise recursion errors when trying to test for attributes on it which ultimately don’t exist (as occurs when the classes are run through Sphinx autodoc), as the unconfigured state of the synonym would put it into an attribute not found loop.¶
References: #4767
[orm] [performance] The optimization applied to selectin loading in #4340 where a JOIN is not needed to eagerly load related items is now applied to many-to-one relationships as well, so that only the related table is queried for a simple join condition. In this case, the related items are queried based on the value of a foreign key column on the parent; if these columns are deferred or otherwise not loaded on any of the parent objects in the collection, the loader falls back to the JOIN method.¶
References: #4775
[engine] [bug] Fixed bug where using reflection function such as MetaData.reflect()
with an Engine object that had execution options applied to it
would fail, as the resulting OptionEngine proxy object failed to
include a .engine attribute used within the reflection routines.¶
References: #4754
[sql] [bug] Adjusted the initialization for Enum to minimize how often it
invokes the .__members__ attribute of a given PEP-435 enumeration
object, to suit the case where this attribute is expensive to invoke, as is
the case for some popular third party enumeration libraries.¶
References: #4758
[sql] [bug] [postgresql] Fixed issue where the array_agg construct in combination with
FunctionElement.filter() would not produce the correct operator
precedence in combination with the array index operator.¶
References: #4760
[sql] [bug] Fixed an unlikely issue where the “corresponding column” routine for unions
and other CompoundSelect objects could return the wrong column in
some overlapping column situtations, thus potentially impacting some ORM
operations when set operations are in use, if the underlying
select() constructs were used previously in other similar kinds of
routines, due to a cached value not being cleared.¶
References: #4747
[postgresql] [usecase] Added support for reflection of indexes on PostgreSQL partitioned tables, which was added to PostgreSQL as of version 11.¶
References: #4771
[postgresql] [usecase] Added support for multidimensional Postgresql array literals via nesting
the postgresql.array object within another one. The
multidimensional array type is detected automatically.
See also
References: #4756
[mysql] [bug] Fixed bug where the special logic to render “NULL” for the
TIMESTAMP datatype when nullable=True would not work if the
column’s datatype were a TypeDecorator or a Variant.
The logic now ensures that it unwraps down to the original
TIMESTAMP so that this special case NULL keyword is correctly
rendered when requested.¶
References: #4743
[mysql] [bug] Enhanced MySQL/MariaDB version string parsing to accommodate for exotic MariaDB version strings where the “MariaDB” word is embedded among other alphanumeric characters such as “MariaDBV1”. This detection is critical in order to correctly accommodate for API features that have split between MySQL and MariaDB such as the “transaction_isolation” system variable.¶
References: #4624
[sqlite] [usecase] Added support for composite (tuple) IN operators with SQLite, by rendering
the VALUES keyword for this backend. As other backends such as DB2 are
known to use the same syntax, the syntax is enabled in the base compiler
using a dialect-level flag tuple_in_values. The change also includes
support for “empty IN tuple” expressions for SQLite when using “in_()”
between a tuple value and an empty set.¶
References: #4766
[mssql] [bug] Ensured that the queries used to reflect indexes and view definitions will
explicitly CAST string parameters into NVARCHAR, as many SQL Server drivers
frequently treat string values, particularly those with non-ascii
characters or larger string values, as TEXT which often don’t compare
correctly against VARCHAR characters in SQL Server’s information schema
tables for some reason. These CAST operations already take place for
reflection queries against SQL Server information_schema. tables but
were missing from three additional queries that are against sys.
tables.¶
References: #4745
[orm] [bug] Fixed a series of related bugs regarding joined table inheritance more than two levels deep, in conjunction with modification to primary key values, where those primary key columns are also linked together in a foreign key relationship as is typical for joined table inheritance. The intermediary table in a three-level inheritance hierarchy will now get its UPDATE if only the primary key value has changed and passive_updates=False (e.g. foreign key constraints not being enforced), whereas before it would be skipped; similarly, with passive_updates=True (e.g. ON UPDATE CASCADE in effect), the third-level table will not receive an UPDATE statement as was the case earlier which would fail since CASCADE already modified it. In a related issue, a relationship linked to a three-level inheritance hierarchy on the primary key of an intermediary table of a joined-inheritance hierarchy will also correctly have its foreign key column updated when the parent object’s primary key is modified, even if that parent object is a subclass of the linked parent class, whereas before these classes would not be counted.¶
References: #4723
[orm] [bug] Fixed bug where the Mapper.all_orm_descriptors accessor would
return an entry for the Mapper itself under the declarative
__mapper___ key, when this is not a descriptor. The .is_attribute
flag that’s present on all InspectionAttr objects is now
consulted, which has also been modified to be True for an association
proxy, as it was erroneously set to False for this object.¶
References: #4729
[orm] [bug] Fixed regression in Query.join() where the aliased=True flag
would not properly apply clause adaptation to filter criteria, if a
previous join were made to the same entity. This is because the adapters
were placed in the wrong order. The order has been reversed so that the
adapter for the most recent aliased=True call takes precedence as was
the case in 1.2 and earlier. This broke the “elementtree” examples among
other things.¶
References: #4704
[orm] [bug] [py3k] Replaced the Python compatbility routines for getfullargspec() with a
fully vendored version from Python 3.3. Originally, Python was emitting
deprecation warnings for this function in Python 3.8 alphas. While this
change was reverted, it was observed that Python 3 implementations for
getfullargspec() are an order of magnitude slower as of the 3.4 series
where it was rewritten against Signature. While Python plans to
improve upon this situation, SQLAlchemy projects for now are using a simple
replacement to avoid any future issues.¶
References: #4674
[orm] [bug] Reworked the attribute mechanics used by AliasedClass to no
longer rely upon calling __getattribute__ on the MRO of the wrapped
class, and to instead resolve the attribute normally on the wrapped class
using getattr(), and then unwrap/adapt that. This allows a greater range
of attribute styles on the mapped class including special __getattr__()
schemes; but it also makes the code simpler and more resilient in general.¶
References: #4694
[sql] [bug] Fixed a series of quoting issues which all stemmed from the concept of the
literal_column() construct, which when being “proxied” through a
subquery to be referred towards by a label that matches its text, the label
would not have quoting rules applied to it, even if the string in the
Label were set up as a quoted_name construct. Not
applying quoting to the text of the Label is a bug because this
text is strictly a SQL identifier name and not a SQL expression, and the
string should not have quotes embedded into it already unlike the
literal_column() which it may be applied towards. The existing
behavior of a non-labeled literal_column() being propagated as is on
the outside of a subquery is maintained in order to help with manual
quoting schemes, although it’s not clear if valid SQL can be generated for
such a construct in any case.¶
References: #4730
[postgresql] [bug] Fixed bug where PostgreSQL dialect could not correctly reflect an ENUM
datatype that has no members, returning a list with None for the
get_enums() call and raising a TypeError when reflecting a column which
has such a datatype. The inspection now returns an empty list.¶
References: #4701
[postgresql] [usecase] Added support for column sorting flags when reflecting indexes for PostgreSQL, including ASC, DESC, NULLSFIRST, NULLSLAST. Also adds this facility to the reflection system in general which can be applied to other dialects in future releases. Pull request courtesy Eli Collins.¶
References: #4717
[orm] [bug] Fixed issue where the AttributeEvents.active_history flag
would not be set for an event listener that propgated to a subclass via the
AttributeEvents.propagate flag. This bug has been present
for the full span of the AttributeEvents system.¶
References: #4695
[orm] [bug] Fixed regression where new association proxy system was still not proxying
hybrid attributes when they made use of the @hybrid_property.expression
decorator to return an alternate SQL expression, or when the hybrid
returned an arbitrary PropComparator, at the expression level.
This involved further generalization of the heuristics used to detect the
type of object being proxied at the level of QueryableAttribute,
to better detect if the descriptor ultimately serves mapped classes or
column expressions.¶
References: #4690
[orm] [bug] Applied the mapper “configure mutex” against the declarative class mapping process, to guard against the race which can occur if mappers are used while dynamic module import schemes are still in the process of configuring mappers for related classes. This does not guard against all possible race conditions, such as if the concurrent import has not yet encountered the dependent classes as of yet, however it guards against as much as possible within the SQLAlchemy declarative process.¶
References: #4686
[orm] [bug] A warning is now emitted for the case where a transient object is being
merged into the session with Session.merge() when that object is
already transient in the Session. This warns for the case where
the object would normally be double-inserted.¶
References: #4647
[orm] [bug] Fixed regression in new relationship m2o comparison logic first introduced at Improvement to the behavior of many-to-one query expressions when comparing to an attribute that is persisted as NULL and is in an un-fetched state in the mapped instance. Since the attribute has no explicit default, it needs to default to NULL when accessed in a persistent setting.¶
References: #4676
[engine] [bug] [postgresql] Moved the “rollback” which occurs during dialect initialization so that it occurs after additional dialect-specific initialize steps, in particular those of the psycopg2 dialect which would inadvertently leave transactional state on the first new connection, which could interfere with some psycopg2-specific APIs which require that no transaction is started. Pull request courtesy Matthew Wilkes.¶
References: #4663
[sql] [bug] Fixed that the GenericFunction class was inadvertently
registering itself as one of the named functions. Pull request courtesy
Adrien Berchet.¶
References: #4653
[sql] [bug] Fixed issue where double negation of a boolean column wouldn’t reset the “NOT” operator.¶
References: #4618
[sql] [bug] The GenericFunction namespace is being migrated so that function
names are looked up in a case-insensitive manner, as SQL functions do not
collide on case sensitive differences nor is this something which would
occur with user-defined functions or stored procedures. Lookups for
functions declared with GenericFunction now use a case
insensitive scheme, however a deprecation case is supported which allows
two or more GenericFunction objects with the same name of
different cases to exist, which will cause case sensitive lookups to occur
for that particular name, while emitting a warning at function registration
time. Thanks to Adrien Berchet for a lot of work on this complicated
feature.¶
References: #4569
[mysql] [bug] Added support for DROP CHECK constraint which is required by MySQL 8.0.16 to drop a CHECK constraint; MariaDB supports plain DROP CONSTRAINT. The logic distinguishes between the two syntaxes by checking the server version string for MariaDB presence. Alembic migrations has already worked around this issue by implementing its own DROP for MySQL / MariaDB CHECK constraints, however this change implements it straight in Core so that its available for general use. Pull request courtesy Hannes Hansen.¶
References: #4650
[mssql] [feature] Added support for SQL Server filtered indexes, via the mssql_where
parameter which works similarly to that of the postgresql_where index
function in the PostgreSQL dialect.
See also
References: #4657
[mssql] [bug] Added error code 20047 to “is_disconnect” for pymssql. Pull request courtesy Jon Schuff.¶
References: #4680
[orm] [bug] Fixed 1.3 regression in new “ambiguous FROMs” query logic introduced in
Query.join() handles ambiguity in deciding the “left” side more explicitly where a Query that explicitly places an entity
in the FROM clause with Query.select_from() and also joins to it
using Query.join() would later cause an “ambiguous FROM” error if
that entity were used in additional joins, as the entity appears twice in
the “from” list of the Query. The fix resolves this ambiguity by
folding the standalone entity into the join that it’s already a part of in
the same way that ultimately happens when the SELECT statement is rendered.¶
References: #4584
[orm] [bug] Adjusted the Query.filter_by() method to not call and()
internally against multiple criteria, instead passing it off to
Query.filter() as a series of criteria, instead of a single criteria.
This allows Query.filter_by() to defer to Query.filter()’s
treatment of variable numbers of clauses, including the case where the list
is empty. In this case, the Query object will not have a
.whereclause, which allows subsequent “no whereclause” methods like
Query.select_from() to behave consistently.¶
References: #4606
[postgresql] [bug] Fixed regression from release 1.3.2 caused by #4562 where a URL that contained only a query string and no hostname, such as for the purposes of specifying a service file with connection information, would no longer be propagated to psycopg2 properly. The change in #4562 has been adjusted to further suit psycopg2’s exact requirements, which is that if there are any connection parameters whatsoever, the “dsn” parameter is no longer required, so in this case the query string parameters are passed alone.¶
References: #4601
[mssql] [bug] Fixed issue in SQL Server dialect where if a bound parameter were present in an ORDER BY expression that would ultimately not be rendered in the SQL Server version of the statement, the parameters would still be part of the execution parameters, leading to DBAPI-level errors. Pull request courtesy Matt Lewellyn.¶
References: #4587
[bug] [pool] Fixed behavioral regression as a result of deprecating the “use_threadlocal”
flag for Pool, where the SingletonThreadPool no longer
makes use of this option which causes the “rollback on return” logic to take
place when the same Engine is used multiple times in the context
of a transaction to connect or implicitly execute, thereby cancelling the
transaction. While this is not the recommended way to work with engines
and connections, it is nonetheless a confusing behavioral change as when
using SingletonThreadPool, the transaction should stay open
regardless of what else is done with the same engine in the same thread.
The use_threadlocal flag remains deprecated however the
SingletonThreadPool now implements its own version of the same
logic.¶
References: #4585
[bug] [ext] Fixed bug where using copy.copy() or copy.deepcopy() on
MutableList would cause the items within the list to be
duplicated, due to an inconsistency in how Python pickle and copy both make
use of __getstate__() and __setstate__() regarding lists. In order
to resolve, a __reduce_ex__ method had to be added to
MutableList. In order to maintain backwards compatibility with
existing pickles based on __getstate__(), the __setstate__() method
remains as well; the test suite asserts that pickles made against the old
version of the class can still be deserialized by the pickle module.¶
References: #4603
[orm] [bug] [ext] Restored instance-level support for plain Python descriptors, e.g.
@property objects, in conjunction with association proxies, in that if
the proxied object is not within ORM scope at all, it gets classified as
“ambiguous” but is proxed directly. For class level access, a basic class
level``__get__()`` now returns the
AmbiguousAssociationProxyInstance directly, rather than raising
its exception, which is the closest approximation to the previous behavior
that returned the AssociationProxy itself that’s possible. Also
improved the stringification of these objects to be more descriptive of
current state.¶
[orm] [bug] Fixed bug where use of with_polymorphic() or other aliased construct
would not properly adapt when the aliased target were used as the
Select.correlate_except() target of a subquery used inside of a
column_property(). This required a fix to the clause adaption
mechanics to properly handle a selectable that shows up in the “correlate
except” list, in a similar manner as which occurs for selectables that show
up in the “correlate” list. This is ultimately a fairly fundamental bug
that has lasted for a long time but it is hard to come across it.¶
References: #4537
[orm] [bug] Fixed regression where a new error message that was supposed to raise when
attempting to link a relationship option to an AliasedClass without using
PropComparator.of_type() would instead raise an AttributeError.
Note that in 1.3, it is no longer valid to create an option path from a
plain mapper relationship to an AliasedClass without using
PropComparator.of_type().¶
References: #4566
[sql] [bug] [documentation] Thanks to TypeEngine methods bind_expression, column_expression work with Variant, type-specific types, we no longer need to rely on recipes that
subclass dialect-specific types directly, TypeDecorator can now
handle all cases. Additionally, the above change made it slightly less
likely that a direct subclass of a base SQLAlchemy type would work as
expected, which could be misleading. Documentation has been updated to use
TypeDecorator for these examples including the PostgreSQL
“ArrayOfEnum” example datatype and direct support for the “subclass a type
directly” has been removed.¶
References: #4580
[postgresql] [feature] Added support for parameter-less connection URLs for the psycopg2 dialect,
meaning, the URL can be passed to create_engine() as
"postgresql+psycopg2://" with no additional arguments to indicate an
empty DSN passed to libpq, which indicates to connect to “localhost” with
no username, password, or database given. Pull request courtesy Julian
Mehnle.¶
References: #4562
[postgresql] [bug] Modified the Select.with_for_update.of parameter so that if a
join or other composed selectable is passed, the individual Table
objects will be filtered from it, allowing one to pass a join() object to
the parameter, as occurs normally when using joined table inheritance with
the ORM. Pull request courtesy Raymond Lu.¶
References: #4550
[mssql] [bug] A commit() is emitted after an isolation level change to SNAPSHOT, as both pyodbc and pymssql open an implicit transaction which blocks subsequent SQL from being emitted in the current transaction.¶
This change is also backported to: 1.2.19
References: #4536
[mssql] [bug] Fixed regression in SQL Server reflection due to #4393 where the
removal of open-ended **kw from the Float datatype caused
reflection of this type to fail due to a “scale” argument being passed.¶
References: #4525
[orm] [feature] The Query.get() method can now accept a dictionary of attribute keys
and values as a means of indicating the primary key value to load; is
particularly useful for composite primary keys. Pull request courtesy
Sanjana S.¶
References: #4316
[orm] [feature] A SQL expression can now be assigned to a primary key attribute for an ORM flush in the same manner as ordinary attributes as described in Embedding SQL Insert/Update Expressions into a Flush where the expression will be evaulated and then returned to the ORM using RETURNING, or in the case of pysqlite, works using the cursor.lastrowid attribute.Requires either a database that supports RETURNING (e.g. Postgresql, Oracle, SQL Server) or pysqlite.¶
References: #3133
[engine] [feature] Revised the formatting for StatementError when stringified. Each
error detail is broken up over multiple newlines instead of spaced out on a
single line. Additionally, the SQL representation now stringifies the SQL
statement rather than using repr(), so that newlines are rendered as is.
Pull request courtesy Nate Clark.
References: #4500
[sql] [bug] The Alias class and related subclasses CTE,
Lateral and TableSample have been reworked so that it is
not possible for a user to construct the objects directly. These constructs
require that the standalone construction function or selectable-bound method
be used to instantiate new objects.¶
References: #4509
[schema] [feature] Added new parameters Table.resolve_fks and
MetaData.reflect.resolve_fks which when set to False will
disable the automatic reflection of related tables encountered in
ForeignKey objects, which can both reduce SQL overhead for omitted
tables as well as avoid tables that can’t be reflected for database-specific
reasons. Two Table objects present in the same MetaData
collection can still refer to each other even if the reflection of the two
tables occurred separately.¶
References: #4517
[orm] [bug] Improved the behavior of orm.with_polymorphic() in conjunction with
loader options, in particular wildcard operations as well as
orm.load_only(). The polymorphic object will be more accurately
targeted so that column-level options on the entity will correctly take
effect.The issue is a continuation of the same kinds of things fixed in
#4468.¶
References: #4469
[bug] [declarative] [orm] Added some helper exceptions that invoke when a mapping based on
AbstractConcreteBase, DeferredReflection, or
AutoMap is used before the mapping is ready to be used, which
contain descriptive information on the class, rather than falling through
into other failure modes that are less informative.¶
References: #4470
[sql] [bug] Fully removed the behavior of strings passed directly as components of a
select() or Query object being coerced to text()
constructs automatically; the warning that has been emitted is now an
ArgumentError or in the case of order_by() / group_by() a CompileError.
This has emitted a warning since version 1.0 however its presence continues
to create concerns for the potential of mis-use of this behavior.
Note that public CVEs have been posted for order_by() / group_by() which are resolved by this commit: CVE-2019-7164 CVE-2019-7548
¶References: #4481
[sql] [bug] Quoting is applied to Function names, those which are usually but
not necessarily generated from the sql.func construct, at compile
time if they contain illegal characters, such as spaces or punctuation. The
names are as before treated as case insensitive however, meaning if the
names contain uppercase or mixed case characters, that alone does not
trigger quoting. The case insensitivity is currently maintained for
backwards compatibility.¶
References: #4467
[sql] [bug] Added “SQL phrase validation” to key DDL phrases that are accepted as plain
strings, including ForeignKeyConstraint.on_delete,
ForeignKeyConstraint.on_update,
ExcludeConstraint.using,
ForeignKeyConstraint.initially, for areas where a series of SQL
keywords only are expected.Any non-space characters that suggest the phrase
would need to be quoted will raise a CompileError. This change
is related to the series of changes committed as part of #4481.¶
References: #4481
[postgresql] [bug] Fixed issue where using an uppercase name for an index type (e.g. GIST, BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to be quoted, rather than rendering it as is. The new behavior converts these types to lowercase and ensures they contain only valid SQL characters.¶
References: #4473
[bug] [ext] Implemented a more comprehensive assignment operation (e.g. “bulk replace”) when using association proxy with sets or dictionaries. Fixes the problem of redundant proxy objects being created to replace the old ones, which leads to excessive events and SQL and in the case of unique constraints will cause the flush to fail.
¶References: #2642
[change] [tests] The test system has removed support for Nose, which is unmaintained for several years and is producing warnings under Python 3. The test suite is currently standardized on Pytest. Pull request courtesy Parth Shandilya.¶
References: #4460
[general] [change] A large change throughout the library has ensured that all objects,
parameters, and behaviors which have been noted as deprecated or legacy now
emit DeprecationWarning warnings when invoked.As the Python 3
interpreter now defaults to displaying deprecation warnings, as well as that
modern test suites based on tools like tox and pytest tend to display
deprecation warnings, this change should make it easier to note what API
features are obsolete. A major rationale for this change is so that long-
deprecated features that nonetheless still see continue to see real world
use can finally be removed in the near future; the biggest example of this
are the SessionExtension and MapperExtension classes as
well as a handful of other pre-event extension hooks, which have been
deprecated since version 0.7 but still remain in the library. Another is
that several major longstanding behaviors are to be deprecated as well,
including the threadlocal engine strategy, the convert_unicode flag, and non
primary mappers.
References: #4393
[orm] [feature] Implemented a new feature whereby the AliasedClass construct can
now be used as the target of a relationship(). This allows the
concept of “non primary mappers” to no longer be necessary, as the
AliasedClass is much easier to configure and automatically inherits
all the relationships of the mapped class, as well as preserves the
ability for loader options to work normally.
References: #4423
[orm] [feature] Added new MapperEvents.before_mapper_configured() event. This
event complements the other “configure” stage mapper events with a per
mapper event that receives each Mapper right before its
configure step, and additionally may be used to prevent or delay the
configuration of specific Mapper objects using a new
return value orm.interfaces.EXT_SKIP. See the
documentation link for an example.
References: #4397
[orm] [bug] Fixed long-standing issue where duplicate collection members would cause a backref to delete the association between the member and its parent object when one of the duplicates were removed, as occurs as a side effect of swapping two objects in one statement.
¶References: #1103
[orm] [bug] Extended the fix first made as part of #3287, where a loader option
made against a subclass using a wildcard would extend itself to include
application of the wildcard to attributes on the super classes as well, to a
“bound” loader option as well, e.g. in an expression like
Load(SomeSubClass).load_only('foo'). Columns that are part of the
parent class of SomeSubClass will also be excluded in the same way as if
the unbound option load_only('foo') were used.¶
References: #4373
[orm] [bug] Improved error messages emitted by the ORM in the area of loader option traversal. This includes early detection of mis-matched loader strategies along with a clearer explanation why these strategies don’t match.¶
References: #4433
[orm] [bug] The “remove” event for collections is now called before the item is removed
in the case of the collection.remove() method, as is consistent with the
behavior for most other forms of collection item removal (such as
__delitem__, replacement under __setitem__). For pop() methods,
the remove event still fires after the operation.¶
[orm] [bug] [engine] Added accessors for execution options to Core and ORM, via
Query.get_execution_options(),
Connection.get_execution_options(),
Engine.get_execution_options(), and
Executable.get_execution_options(). PR courtesy Daniel Lister.¶
References: #4464
[orm] [bug] Fixed issue in association proxy due to #3423 which caused the use
of custom PropComparator objects with hybrid attributes, such as
the one demonstrated in the dictlike-polymorphic example to not
function within an association proxy. The strictness that was added in
#3423 has been relaxed, and additional logic to accommodate for
an association proxy that links to a custom hybrid have been added.¶
References: #4446
[orm] [bug] Implemented the .get_history() method, which also implies availability
of AttributeState.history, for synonym() attributes.
Previously, trying to access attribute history via a synonym would raise an
AttributeError.¶
References: #3777
[orm] [change] Added a new function close_all_sessions() which takes
over the task of the Session.close_all() method, which
is now deprecated as this is confusing as a classmethod.
Pull request courtesy Augustin Trancart.¶
References: #4412
[orm declarative] [bug] Added a __clause_element__() method to ColumnProperty which
can allow the usage of a not-fully-declared column or deferred attribute in
a declarative mapped class slightly more friendly when it’s used in a
constraint or other column-oriented scenario within the class declaration,
though this still can’t work in open-ended expressions; prefer to call the
ColumnProperty.expression attribute if receiving TypeError.¶
References: #4372
[engine] [feature] Added public accessor QueuePool.timeout() that returns the configured
timeout for a QueuePool object. Pull request courtesy Irina Delamare.¶
References: #3689
[engine] [change] The “threadlocal” engine strategy which has been a legacy feature of
SQLAlchemy since around version 0.2 is now deprecated, along with the
Pool.threadlocal parameter of Pool which has no
effect in most modern use cases.
References: #4393
[sql] [feature] Amended the AnsiFunction class, the base of common SQL
functions like CURRENT_TIMESTAMP, to accept positional arguments
like a regular ad-hoc function. This to suit the case that many of
these functions on specific backends accept arguments such as
“fractional seconds” precision and such. If the function is created
with arguments, it renders the parenthesis and the arguments. If
no arguments are present, the compiler generates the non-parenthesized form.¶
References: #4386
[sql] [change] The create_engine.convert_unicode and
String.convert_unicode parameters have been deprecated. These
parameters were built back when most Python DBAPIs had little to no support
for Python Unicode objects, and SQLAlchemy needed to take on the very
complex task of marshalling data and SQL strings between Unicode and
bytestrings throughout the system in a performant way. Thanks to Python 3,
DBAPIs were compelled to adapt to Unicode-aware APIs and today all DBAPIs
supported by SQLAlchemy support Unicode natively, including on Python 2,
allowing this long-lived and very complicated feature to finally be (mostly)
removed. There are still of course a few Python 2 edge cases where
SQLAlchemy has to deal with Unicode however these are handled automatically;
in modern use, there should be no need for end-user interaction with these
flags.
References: #4393
[mssql] [bug] The literal_processor for the Unicode and
UnicodeText datatypes now render an N character in front of
the literal string expression as required by SQL Server for Unicode string
values rendered in SQL expressions.¶
References: #4442
[orm] [feature] Added new feature Query.only_return_tuples(). Causes the
Query object to return keyed tuple objects unconditionally even
if the query is against a single entity. Pull request courtesy Eric
Atkin.¶
This change is also backported to: 1.2.5
[orm] [feature] Added new flag Session.bulk_save_objects.preserve_order to the
Session.bulk_save_objects() method, which defaults to True. When set
to False, the given mappings will be grouped into inserts and updates per
each object type, to allow for greater opportunities to batch common
operations together. Pull request courtesy Alessandro Cucci.¶
[orm] [feature] The “selectin” loader strategy now omits the JOIN in the case of a simple
one-to-many load, where it instead relies loads only from the related
table, relying upon the foreign key columns of the related table in order
to match up to primary keys in the parent table. This optimization can be
disabled by setting the relationship.omit_join flag to False.
Many thanks to Jayson Reis for the efforts on this.
References: #4340
[orm] [feature] Added .info dictionary to the InstanceState class, the object
that comes from calling inspect() on a mapped object.
References: #4257
[orm] [bug] Fixed bug where use of Lateral construct in conjunction with
Query.join() as well as Query.select_entity_from() would not
apply clause adaption to the right side of the join. “lateral” introduces
the use case of the right side of a join being correlatable. Previously,
adaptation of this clause wasn’t considered. Note that in 1.2 only,
a selectable introduced by Query.subquery() is still not adapted
due to #4304; the selectable needs to be produced by the
select() function to be the right side of the “lateral” join.¶
This change is also backported to: 1.2.12
References: #4334
[orm] [bug] Fixed issue regarding passive_deletes=”all”, where the foreign key attribute of an object is maintained with its value even after the object is removed from its parent collection. Previously, the unit of work would set this to NULL even though passive_deletes indicated it should not be modified.
¶References: #3844
[orm] [bug] Improved the behavior of a relationship-bound many-to-one object expression
such that the retrieval of column values on the related object are now
resilient against the object being detached from its parent
Session, even if the attribute has been expired. New features
within the InstanceState are used to memoize the last known value
of a particular column attribute before its expired, so that the expression
can still evaluate when the object is detached and expired at the same
time. Error conditions are also improved using modern attribute state
features to produce more specific messages as needed.
References: #4359
[orm] [bug] [mysql] [postgresql] The ORM now doubles the “FOR UPDATE” clause within the subquery that renders in conjunction with joined eager loading in some cases, as it has been observed that MySQL does not lock the rows from a subquery. This means the query renders with two FOR UPDATE clauses; note that on some backends such as Oracle, FOR UPDATE clauses on subqueries are silently ignored since they are unnecessary. Additionally, in the case of the “OF” clause used primarily with PostgreSQL, the FOR UPDATE is rendered only on the inner subquery when this is used so that the selectable can be targeted to the table within the SELECT statement.
¶References: #4246
[orm] [bug] Refactored Query.join() to further clarify the individual components
of structuring the join. This refactor adds the ability for
Query.join() to determine the most appropriate “left” side of the
join when there is more than one element in the FROM list or the query is
against multiple entities. If more than one FROM/entity matches, an error
is raised that asks for an ON clause to be specified to resolve the
ambiguity. In particular this targets the regression we saw in
#4363 but is also of general use. The codepaths within
Query.join() are now easier to follow and the error cases are
decided more specifically at an earlier point in the operation.
References: #4365
[orm] [bug] Fixed long-standing issue in Query where a scalar subquery such
as produced by Query.exists(), Query.as_scalar() and other
derivations from Query.statement would not correctly be adapted
when used in a new Query that required entity adaptation, such as
when the query were turned into a union, or a from_self(), etc. The change
removes the “no adaptation” annotation from the select() object
produced by the Query.statement accessor.¶
References: #4304
[orm] [bug] An informative exception is re-raised when a primary key value is not
sortable in Python during an ORM flush under Python 3, such as an Enum
that has no __lt__() method; normally Python 3 raises a TypeError
in this case. The flush process sorts persistent objects by primary key
in Python so the values must be sortable.¶
References: #4232
[orm] [bug] Removed the collection converter used by the MappedCollection
class. This converter was used only to assert that the incoming dictionary
keys matched that of their corresponding objects, and only during a bulk set
operation. The converter can interfere with a custom validator or
AttributeEvents.bulk_replace() listener that wants to convert
incoming values further. The TypeError which would be raised by this
converter when an incoming key didn’t match the value is removed; incoming
values during a bulk assignment will be keyed to their value-generated key,
and not the key that’s explicitly present in the dictionary.
Overall, @converter is superseded by the
AttributeEvents.bulk_replace() event handler added as part of
#3896.
References: #3604
[orm] [bug] Added new behavior to the lazy load that takes place when the “old” value of
a many-to-one is retrieved, such that exceptions which would be raised due
to either lazy="raise" or a detached session error are skipped.
References: #4353
[orm] [bug] A long-standing oversight in the ORM, the __delete__ method for a many-
to-one relationship was non-functional, e.g. for an operation such as del
a.b. This is now implemented and is equivalent to setting the attribute
to None.
See also
References: #4354
[bug] [declarative] [orm] Fixed bug where declarative would not update the state of the
Mapper as far as what attributes were present, when additional
attributes were added or removed after the mapper attribute collections had
already been called and memoized. Additionally, a NotImplementedError
is now raised if a fully mapped attribute (e.g. column, relationship, etc.)
is deleted from a class that is currently mapped, since the mapper will not
function correctly if the attribute has been removed.¶
References: #4133
[engine] [feature] Added new “lifo” mode to QueuePool, typically enabled by setting
the flag create_engine.pool_use_lifo to True. “lifo” mode
means the same connection just checked in will be the first to be checked
out again, allowing excess connections to be cleaned up from the server
side during periods of the pool being only partially utilized. Pull request
courtesy Taem Park.
[sql] [feature] Refactored SQLCompiler to expose a
SQLCompiler.group_by_clause() method similar to the
SQLCompiler.order_by_clause() and SQLCompiler.limit_clause()
methods, which can be overridden by dialects to customize how GROUP BY
renders. Pull request courtesy Samuel Chou.¶
This change is also backported to: 1.2.13
[sql] [feature] Added Sequence to the “string SQL” system that will render a
meaningful string expression ("<next sequence value: my_sequence>")
when stringifying without a dialect a statement that includes a “sequence
nextvalue” expression, rather than raising a compilation error.¶
References: #4144
[sql] [feature] Added new naming convention tokens column_0N_name, column_0_N_name,
etc., which will render the names / keys / labels for all columns referenced
by a particular constraint in a sequence. In order to accommodate for the
length of such a naming convention, the SQL compiler’s auto-truncation
feature now applies itself to constraint names as well, which creates a
shortened, deterministically generated name for the constraint that will
apply to a target backend without going over the character limit of that
backend.
The change also repairs two other issues. One is that the column_0_key
token wasn’t available even though this token was documented, the other was
that the referred_column_0_name token would inadvertently render the
.key and not the .name of the column if these two values were
different.
References: #3989
[sql] [feature] Added new logic to the “expanding IN” bound parameter feature whereby if the given list is empty, a special “empty set” expression that is specific to different backends is generated, thus allowing IN expressions to be fully dynamic including empty IN expressions.
¶References: #4271
[sql] [feature] The Python builtin dir() is now supported for a SQLAlchemy “properties”
object, such as that of a Core columns collection (e.g. .c),
mapper.attrs, etc. Allows iPython autocompletion to work as well.
Pull request courtesy Uwe Korn.¶
[sql] [feature] Added new feature FunctionElement.as_comparison() which allows a SQL
function to act as a binary comparison operation that can work within the
ORM.
References: #3831
[sql] [bug] Added “like” based operators as “comparison” operators, including
ColumnOperators.startswith() ColumnOperators.endswith()
ColumnOperators.ilike() ColumnOperators.notilike() among many
others, so that all of these operators can be the basis for an ORM
“primaryjoin” condition.¶
References: #4302
[sql] [bug] Fixed issue with TypeEngine.bind_expression() and
TypeEngine.column_expression() methods where these methods would not
work if the target type were part of a Variant, or other target
type of a TypeDecorator. Additionally, the SQL compiler now
calls upon the dialect-level implementation when it renders these methods
so that dialects can now provide for SQL-level processing for built-in
types.
See also
TypeEngine methods bind_expression, column_expression work with Variant, type-specific types
References: #3981
[postgresql] [feature] Added new PG type postgresql.REGCLASS which assists in casting
table names to OID values. Pull request courtesy Sebastian Bank.¶
This change is also backported to: 1.2.7
References: #4160
[postgresql] [feature] Added rudimental support for reflection of PostgreSQL partitioned tables, e.g. that relkind=’p’ is added to reflection queries that return table information.
¶References: #4237
[mysql] [feature] Support added for the “WITH PARSER” syntax of CREATE FULLTEXT INDEX
in MySQL, using the mysql_with_parser keyword argument. Reflection
is also supported, which accommodates MySQL’s special comment format
for reporting on this option as well. Additionally, the “FULLTEXT” and
“SPATIAL” index prefixes are now reflected back into the mysql_prefix
index option.¶
References: #4219
[mysql] [feature] Added support for the parameters in an ON DUPLICATE KEY UPDATE statement on MySQL to be ordered, since parameter order in a MySQL UPDATE clause is significant, in a similar manner as that described at Parameter-Ordered Updates. Pull request courtesy Maxim Bublis.
¶[mysql] [feature] The “pre-ping” feature of the connection pool now uses
the ping() method of the DBAPI connection in the case of
mysqlclient, PyMySQL and mysql-connector-python. Pull request
courtesy Maxim Bublis.
[sqlite] [feature] Added support for SQLite’s json functionality via the new
SQLite implementation for types.JSON, sqlite.JSON.
The name used for the type is JSON, following an example found at
SQLite’s own documentation. Pull request courtesy Ilja Everilä.
See also
References: #3850
[sqlite] [feature] Implemented the SQLite ON CONFLICT clause as understood at the DDL
level, e.g. for primary key, unique, and CHECK constraints as well as
specified on a Column to satisfy inline primary key and NOT NULL.
Pull request courtesy Denis Kataev.
References: #4360
[mssql] [feature] Added fast_executemany=True parameter to the SQL Server pyodbc dialect,
which enables use of pyodbc’s new performance feature of the same name
when using Microsoft ODBC drivers.
See also
References: #4158
[mssql] [bug] Deprecated the use of Sequence with SQL Server in order to affect
the “start” and “increment” of the IDENTITY value, in favor of new
parameters mssql_identity_start and mssql_identity_increment which
set these parameters directly. Sequence will be used to generate
real CREATE SEQUENCE DDL with SQL Server in a future release.
References: #4362
[oracle] [feature] Added a new event currently used only by the cx_Oracle dialect,
DialectEvents.setiputsizes(). The event passes a dictionary of
BindParameter objects to DBAPI-specific type objects that will be
passed, after conversion to parameter names, to the cx_Oracle
cursor.setinputsizes() method. This allows both visibility into the
setinputsizes process as well as the ability to alter the behavior of what
datatypes are passed to this method.
This change is also backported to: 1.2.9
References: #4290
[oracle] [bug] Updated the parameters that can be sent to the cx_Oracle DBAPI to both allow for all current parameters as well as for future parameters not added yet. In addition, removed unused parameters that were deprecated in version 1.2, and additionally we are now defaulting “threaded” to False.
¶References: #4369
[oracle] [bug] The Oracle dialect will no longer use the NCHAR/NCLOB datatypes
represent generic unicode strings or clob fields in conjunction with
Unicode and UnicodeText unless the flag
use_nchar_for_unicode=True is passed to create_engine() -
this includes CREATE TABLE behavior as well as setinputsizes() for
bound parameters. On the read side, automatic Unicode conversion under
Python 2 has been added to CHAR/VARCHAR/CLOB result rows, to match the
behavior of cx_Oracle under Python 3. In order to mitigate the performance
hit under Python 2, SQLAlchemy’s very performant (when C extensions
are built) native Unicode handlers are used under Python 2.
References: #4242
[feature] [ext] Added new attribute Query.lazy_loaded_from which is populated
with an InstanceState that is using this Query in
order to lazy load a relationship. The rationale for this is that
it serves as a hint for the horizontal sharding feature to use, such that
the identity token of the state can be used as the default identity token
to use for the query within id_chooser().¶
This change is also backported to: 1.2.9
References: #4243
[feature] [ext] Added new feature BakedQuery.to_query(), which allows for a
clean way of using one BakedQuery as a subquery inside of another
BakedQuery without needing to refer explicitly to a
Session.¶
References: #4318
[feature] [ext] The AssociationProxy now has standard column comparison operations
such as ColumnOperators.like() and
ColumnOperators.startswith() available when the target attribute is a
plain column - the EXISTS expression that joins to the target table is
rendered as usual, but the column expression is then use within the WHERE
criteria of the EXISTS. Note that this alters the behavior of the
.contains() method on the association proxy to make use of
ColumnOperators.contains() when used on a column-based attribute.
References: #4351
[feature] [ext] Added support for bulk Query.update() and Query.delete()
to the ShardedQuery class within the horizontal sharding
extension. This also adds an additional expansion hook to the
bulk update/delete methods Query._execute_crud().
References: #4196
[bug] [ext] Reworked AssociationProxy to store state that’s specific to a
parent class in a separate object, so that a single
AssociationProxy can serve for multiple parent classes, as is
intrinsic to inheritance, without any ambiguity in the state returned by it.
A new method AssociationProxy.for_class() is added to allow
inspection of class-specific state.
References: #3423
[bug] [ext] The long-standing behavior of the association proxy collection maintaining only a weak reference to the parent object is reverted; the proxy will now maintain a strong reference to the parent for as long as the proxy collection itself is also in memory, eliminating the “stale association proxy” error. This change is being made on an experimental basis to see if any use cases arise where it causes side effects.
See also
References: #4268
[bug] [ext] Fixed multiple issues regarding de-association of scalar objects with the
association proxy. del now works, and additionally a new flag
AssociationProxy.cascade_scalar_deletes is added, which when
set to True indicates that setting a scalar attribute to None or
deleting via del will also set the source association to None.
References: #4308