From 20aa4ad0f1ca6036792bdba1fdaa959128649f8f Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Mon, 31 May 2021 13:57:42 -0400 Subject: [PATCH 1/9] Fixups for whatsnew 1.3.0 --- doc/source/whatsnew/v1.3.0.rst | 454 ++++++++++++++++----------------- 1 file changed, 227 insertions(+), 227 deletions(-) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index ea9017da8a2f9..c39a5a1836e19 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -28,7 +28,7 @@ Enhancements Custom HTTP(s) headers when reading csv or json files ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -When reading from a remote URL that is not handled by fsspec (ie. HTTP and +When reading from a remote URL that is not handled by fsspec (e.g. HTTP and HTTPS) the dictionary passed to ``storage_options`` will be used to create the headers included in the request. This can be used to control the User-Agent header or send other custom headers (:issue:`36688`). @@ -50,7 +50,7 @@ Read and write XML documents We added I/O support to read and render shallow versions of `XML`_ documents with :func:`pandas.read_xml` and :meth:`DataFrame.to_xml`. Using `lxml`_ as parser, -both XPath 1.0 and XSLT 1.0 is available. (:issue:`27554`) +both XPath 1.0 and XSLT 1.0 are available. (:issue:`27554`) .. _XML: https://www.w3.org/standards/xml/core .. _lxml: https://lxml.de @@ -110,8 +110,10 @@ both XPath 1.0 and XSLT 1.0 is available. (:issue:`27554`) For more, see :ref:`io.xml` in the user guide on IO tools. -Styler Upgrades -^^^^^^^^^^^^^^^ +.. _whatsnew_130.styler_upgrades: + +Styler Improvements +^^^^^^^^^^^^^^^^^^^ We provided some focused development on :class:`.Styler`, including altering methods to accept more universal CSS language for arguments, such as ``'color:red;'`` instead of @@ -119,32 +121,32 @@ to accept more universal CSS language for arguments, such as ``'color:red;'`` in to allow custom CSS highlighting instead of default background coloring (:issue:`40242`). Enhancements to other built-in methods include extending the :meth:`.Styler.background_gradient` method to shade elements based on a given gradient map and not be restricted only to -values in the DataFrame (:issue:`39930` :issue:`22727` :issue:`28901`). Additional +values in the DataFrame (:issue:`39930`, :issue:`22727`, :issue:`28901`). Additional built-in methods such as :meth:`.Styler.highlight_between`, :meth:`.Styler.highlight_quantile` -and :math:`.Styler.text_gradient` have been added (:issue:`39821`, :issue:`40926`, :issue:`41098`). +and :meth:`.Styler.text_gradient` have been added (:issue:`39821`, :issue:`40926`, :issue:`41098`). -The :meth:`.Styler.apply` now consistently allows functions with ``ndarray`` output to -allow more flexible development of UDFs when ``axis`` is ``None`` ``0`` or ``1`` (:issue:`39393`). +The :meth:`.Styler.apply` method now consistently accepts functions with ``ndarray`` output to +allow more flexible development of UDFs when ``axis`` is ``None``, ``0``, or ``1`` (:issue:`39393`). -:meth:`.Styler.set_tooltips` is a new method that allows adding on hover tooltips to +:meth:`.Styler.set_tooltips` is a new method that allows adding hover tooltips to enhance interactive displays (:issue:`35643`). :meth:`.Styler.set_td_classes`, which was recently introduced in v1.2.0 (:issue:`36159`) to allow adding specific CSS classes to data cells, has been made as performant as :meth:`.Styler.apply` and :meth:`.Styler.applymap` (:issue:`40453`), if not more performant in some cases. The overall performance of HTML render times has been considerably improved to -match :meth:`DataFrame.to_html` (:issue:`39952` :issue:`37792` :issue:`40425`). +match :meth:`DataFrame.to_html` (:issue:`39952`, :issue:`37792`, :issue:`40425`). -The :meth:`.Styler.format` has had upgrades to easily format missing data, -precision, and perform HTML escaping (:issue:`40437` :issue:`40134`). There have been numerous other bug fixes to -properly format HTML and eliminate some inconsistencies (:issue:`39942` :issue:`40356` :issue:`39807` :issue:`39889` :issue:`39627`) +The :meth:`.Styler.format` has been improved to easily format missing data, +precision, and perform HTML escaping (:issue:`40437`, :issue:`40134`). There have been numerous other bug fixes to +properly format HTML and eliminate some inconsistencies (:issue:`39942`, :issue:`40356`, :issue:`39807`, :issue:`39889`, :issue:`39627`). -:class:`.Styler` has also been compatible with non-unique index or columns, at least for as many features as are fully compatible, others made only partially compatible (:issue:`41269`). -One also has greater control of the display through separate sparsification of the index or columns, using the new 'styler' options context (:issue:`41142`). +Many features of the :class:`.Styler` class are now either partially or fully usable on a DataFrame with a non-unique indexes or columns (:issue:`41269`). +One also has greater control of the display through separate sparsification of the index or columns using the :ref:`new 'styler' options context ` (:issue:`41142`). -We have added an extension to allow LaTeX styling as an alternative to CSS styling and a method :meth:`.Styler.to_latex` -which renders the necessary LaTeX format including built-up styles. An additional file io function :meth:`Styler.to_html` has been added for convenience (:issue:`40312`). +We have added an extension to allow LaTeX styling as an alternative to CSS styling and the method :meth:`.Styler.to_latex` +that renders the necessary LaTeX format including built-up styles. An additional file IO function :meth:`.Styler.to_html` has been added for convenience (:issue:`40312`). -Documentation has also seen major revisions in light of new features (:issue:`39720` :issue:`39317` :issue:`40493`) +The :ref:`Styler documentation <../user_guide/style.ipynb>`_ has also seen major revisions in light of new features (:issue:`39720`, :issue:`39317`, :issue:`40493`). .. _whatsnew_130.dataframe_honors_copy_with_dict: @@ -152,7 +154,7 @@ DataFrame constructor honors ``copy=False`` with dict ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When passing a dictionary to :class:`DataFrame` with ``copy=False``, -a copy will no longer be made (:issue:`32960`) +a copy will no longer be made (:issue:`32960`). .. ipython:: python @@ -170,10 +172,10 @@ a copy will no longer be made (:issue:`32960`) The default behavior when not passing ``copy`` will remain unchanged, i.e. a copy will be made. -Centered Datetime-Like Rolling Windows +Centered datetime-like rolling windows ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -When performing rolling calculations on :class:`DataFrame` and :class:`Series` +When performing rolling calculations on DataFrame and Series objects with a datetime-like index, a centered datetime-like window can now be used (:issue:`38780`). For example: @@ -192,40 +194,40 @@ For example: Other enhancements ^^^^^^^^^^^^^^^^^^ -- :class:`Rolling` and :class:`Expanding` now support a ``method`` argument with a ``'table'`` option that performs the windowing operation over an entire :class:`DataFrame`. See ref:`window.overview` for performance and functional benefits (:issue:`15095`, :issue:`38995`) +- :meth:`DataFrame.rolling`, :meth:`Series.rolling`, :meth:`DataFrame.expanding` and :meth:`Series.expanding` now support a ``method`` argument with a ``'table'`` option that performs the windowing operation over an entire DataFrame. See :ref:`Window Overview ` for performance and functional benefits (:issue:`15095`, :issue:`38995`) - Added :meth:`MultiIndex.dtypes` (:issue:`37062`) -- Added ``end`` and ``end_day`` options for ``origin`` in :meth:`DataFrame.resample` (:issue:`37804`) -- Improve error message when ``usecols`` and ``names`` do not match for :func:`read_csv` and ``engine="c"`` (:issue:`29042`) -- Improved consistency of error message when passing an invalid ``win_type`` argument in :class:`Window` (:issue:`15969`) -- :func:`pandas.read_sql_query` now accepts a ``dtype`` argument to cast the columnar data from the SQL database based on user input (:issue:`10285`) +- Added ``end`` and ``end_day`` options for the ``origin`` argument in :meth:`DataFrame.resample` (:issue:`37804`) +- Improved error message when ``usecols`` and ``names`` do not match for :func:`read_csv` and ``engine="c"`` (:issue:`29042`) +- Improved consistency of error messages when passing an invalid ``win_type`` argument in :ref:`Window methods ` (:issue:`15969`) +- :func:`read_sql_query` now accepts a ``dtype`` argument to cast the columnar data from the SQL database based on user input (:issue:`10285`) - Improved integer type mapping from pandas to SQLAlchemy when using :meth:`DataFrame.to_sql` (:issue:`35076`) - :func:`to_numeric` now supports downcasting of nullable ``ExtensionDtype`` objects (:issue:`33013`) -- Add support for dict-like names in :class:`MultiIndex.set_names` and :class:`MultiIndex.rename` (:issue:`20421`) -- :func:`pandas.read_excel` can now auto detect .xlsb files and older .xls files (:issue:`35416`, :issue:`41225`) -- :class:`pandas.ExcelWriter` now accepts an ``if_sheet_exists`` parameter to control the behaviour of append mode when writing to existing sheets (:issue:`40230`) -- :meth:`.Rolling.sum`, :meth:`.Expanding.sum`, :meth:`.Rolling.mean`, :meth:`.Expanding.mean`, :meth:`.ExponentialMovingWindow.mean`, :meth:`.Rolling.median`, :meth:`.Expanding.median`, :meth:`.Rolling.max`, :meth:`.Expanding.max`, :meth:`.Rolling.min`, and :meth:`.Expanding.min` now support ``Numba`` execution with the ``engine`` keyword (:issue:`38895`, :issue:`41267`) +- Add support for dict-like names in :meth:`Index.set_names` and :meth:`.Index.rename` in the case of a :class:`MultiIndex` (:issue:`20421`) +- :func:`read_excel` can now auto-detect .xlsb files and older .xls files (:issue:`35416`, :issue:`41225`) +- :class:`ExcelWriter` now accepts an ``if_sheet_exists`` parameter to control the behaviour of append mode when writing to existing sheets (:issue:`40230`) +- :meth:`.Rolling.sum`, :meth:`.Expanding.sum`, :meth:`.Rolling.mean`, :meth:`.Expanding.mean`, :meth:`.ExponentialMovingWindow.mean`, :meth:`.Rolling.median`, :meth:`.Expanding.median`, :meth:`.Rolling.max`, :meth:`.Expanding.max`, :meth:`.Rolling.min`, and :meth:`.Expanding.min` now support `Numba `_ execution with the ``engine`` keyword (:issue:`38895`, :issue:`41267`) - :meth:`DataFrame.apply` can now accept NumPy unary operators as strings, e.g. ``df.apply("sqrt")``, which was already the case for :meth:`Series.apply` (:issue:`39116`) - :meth:`DataFrame.apply` can now accept non-callable DataFrame properties as strings, e.g. ``df.apply("size")``, which was already the case for :meth:`Series.apply` (:issue:`39116`) -- :meth:`DataFrame.applymap` can now accept kwargs to pass on to func (:issue:`39987`) -- Disallow :class:`DataFrame` indexer for ``iloc`` for :meth:`Series.__getitem__` and :meth:`DataFrame.__getitem__`, (:issue:`39004`) +- :meth:`DataFrame.applymap` can now accept kwargs to pass on to the user-provided ``func`` (:issue:`39987`) +- Passing a :class:`DataFrame` indexer to ``iloc`` is now disallowed for :meth:`Series.__getitem__` and :meth:`DataFrame.__getitem__` (:issue:`39004`) - :meth:`Series.apply` can now accept list-like or dictionary-like arguments that aren't lists or dictionaries, e.g. ``ser.apply(np.array(["sum", "mean"]))``, which was already the case for :meth:`DataFrame.apply` (:issue:`39140`) -- :meth:`DataFrame.plot.scatter` can now accept a categorical column as the argument to ``c`` (:issue:`12380`, :issue:`31357`) +- :meth:`DataFrame.plot.scatter` can now accept a categorical column for the argument ``c`` (:issue:`12380`, :issue:`31357`) - :meth:`.Styler.set_tooltips` allows on hover tooltips to be added to styled HTML dataframes (:issue:`35643`, :issue:`21266`, :issue:`39317`, :issue:`39708`, :issue:`40284`) -- :meth:`.Styler.set_table_styles` amended to optionally allow certain css-string input arguments (:issue:`39564`) -- :meth:`.Styler.apply` now more consistently accepts ndarray function returns, i.e. in all cases for ``axis`` is ``0, 1 or None`` (:issue:`39359`) -- :meth:`.Styler.apply` and :meth:`.Styler.applymap` now raise errors if wrong format CSS is passed on render (:issue:`39660`) -- :meth:`.Styler.format` adds keyword argument ``escape`` for optional HTML escaping (:issue:`40437`) -- :meth:`.Styler.background_gradient` now allows the ability to supply a specific gradient map (:issue:`22727`) +- :meth:`.Styler.set_table_styles` amended to optionally allow certain CSS input arguments as strings (:issue:`39564`) +- :meth:`.Styler.apply` now accepts functions that return an ``ndarray`` when ``axis=None``, consistent with the ``axis=0`` and ``axis=1`` behavior (:issue:`39359`) +- :meth:`.Styler.apply` and :meth:`.Styler.applymap` now raise errors if incorrectly formatted CSS is passed (:issue:`39660`) +- :meth:`.Styler.format` now accepts the keyword argument ``escape`` for optional HTML escaping (:issue:`40437`) +- :meth:`.Styler.background_gradient` has gained the argument ``gmap`` to supply a specific gradient map (:issue:`22727`) - :meth:`.Styler.clear` now clears :attr:`Styler.hidden_index` and :attr:`Styler.hidden_columns` as well (:issue:`40484`) - Builtin highlighting methods in :class:`Styler` have a more consistent signature and css customisability (:issue:`40242`) -- :meth:`.Styler.highlight_between` added to list of builtin styling methods (:issue:`39821`) -- :meth:`Series.loc.__getitem__` and :meth:`Series.loc.__setitem__` with :class:`MultiIndex` now raising helpful error message when indexer has too many dimensions (:issue:`35349`) -- :meth:`pandas.read_stata` and :class:`StataReader` support reading data from compressed files. -- Add support for parsing ``ISO 8601``-like timestamps with negative signs to :meth:`pandas.Timedelta` (:issue:`37172`) -- Add support for unary operators in :class:`FloatingArray` (:issue:`38749`) +- Added the method :meth:`.Styler.highlight_between` (:issue:`39821`) +- :meth:`Series.loc` now raises a helpful error message when the Series has a :class:`MultiIndex` and the indexer has too many dimensions (:issue:`35349`) +- :func:`read_stata` now supports reading data from compressed files (:issue:`26599`) +- Added support for parsing ``ISO 8601``-like timestamps with negative signs to :class:`Timedelta` (:issue:`37172`) +- Added support for unary operators in :class:`FloatingArray` (:issue:`38749`) - :class:`RangeIndex` can now be constructed by passing a ``range`` object directly e.g. ``pd.RangeIndex(range(3))`` (:issue:`12067`) -- :meth:`round` being enabled for the nullable integer and floating dtypes (:issue:`38844`) -- :meth:`pandas.read_csv` and :meth:`pandas.read_json` expose the argument ``encoding_errors`` to control how encoding errors are handled (:issue:`39450`) +- :meth:`Series.round` and :meth:`DataFrame.round` now work with nullable integer and floating dtypes (:issue:`38844`) +- :meth:`read_csv` and :meth:`read_json` expose the argument ``encoding_errors`` to control how encoding errors are handled (:issue:`39450`) - :meth:`.GroupBy.any` and :meth:`.GroupBy.all` use Kleene logic with nullable data types (:issue:`37506`) - :meth:`.GroupBy.any` and :meth:`.GroupBy.all` return a ``BooleanDtype`` for columns with nullable data types (:issue:`33449`) - :meth:`.GroupBy.rank` now supports object-dtype data (:issue:`38278`) @@ -246,9 +248,9 @@ These are bug fixes that might have notable behavior changes. ``Categorical.unique`` now always maintains same dtype as original ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Previously, when calling :meth:`~Categorical.unique` with categorical data, unused categories in the new array -would be removed, meaning that the dtype of the new array would be different than the -original, if some categories are not present in the unique array (:issue:`18291`) +Previously, when calling :meth:`Categorical.unique` with categorical data, unused categories in the new array +would be removed, making the dtype of the new array different than the +original (:issue:`18291`) As an example of this, given: @@ -404,7 +406,7 @@ In pandas 1.3.0, ``df`` continues to share data with ``values`` .. _whatsnew_130.notable_bug_fixes.setitem_never_inplace: -Never Operate Inplace When Setting ``frame[keys] = values`` +Never operate inplace when setting ``frame[keys] = values`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When setting multiple columns using ``frame[keys] = values`` new arrays will @@ -439,11 +441,11 @@ In the new behavior, we get a new array, and retain an integer-dtyped ``5``: .. _whatsnew_130.notable_bug_fixes.setitem_with_bool_casting: -Consistent Casting With Setting Into Boolean Series +Consistent casting with setting into Boolean Series ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Setting non-boolean values into a :class:`Series with ``dtype=bool`` consistently -cast to ``dtype=object`` (:issue:`38709`) +Setting non-boolean values into a :class:`Series` with ``dtype=bool`` now consistently +casts to ``dtype=object`` (:issue:`38709`) .. ipython:: python @@ -514,7 +516,7 @@ The group-by column will now be dropped from the result of a Removed artificial truncation in rolling variance and standard deviation ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -:meth:`core.window.Rolling.std` and :meth:`core.window.Rolling.var` will no longer +:meth:`.Rolling.std` and :meth:`.Rolling.var` will no longer artificially truncate results that are less than ``~1e-8`` and ``~1e-15`` respectively to zero (:issue:`37051`, :issue:`40448`, :issue:`39872`). @@ -530,7 +532,7 @@ However, floating point artifacts may now exist in the results when rolling over GroupBy.rolling with MultiIndex no longer drops levels in the result ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -:class:`core.window.rolling.RollingGroupby` will no longer drop levels of a :class:`DataFrame` +:meth:`GroupBy.rolling` will no longer drop levels of a :class:`DataFrame` with a :class:`MultiIndex` in the result. This can lead to a perceived duplication of levels in the resulting :class:`MultiIndex`, but this change restores the behavior that was present in version 1.1.3 (:issue:`38787`, :issue:`38523`). @@ -639,12 +641,11 @@ See :ref:`install.dependencies` and :ref:`install.optional_dependencies` for mor Other API changes ^^^^^^^^^^^^^^^^^ -- Partially initialized :class:`CategoricalDtype` (i.e. those with ``categories=None`` objects will no longer compare as equal to fully initialized dtype objects. -- Accessing ``_constructor_expanddim`` on a :class:`DataFrame` and ``_constructor_sliced`` on a :class:`Series` now raise an ``AttributeError``. Previously a ``NotImplementedError`` was raised (:issue:`38782`) -- Added new ``engine`` and ``**engine_kwargs`` parameters to :meth:`DataFrame.to_sql` to support other future "SQL engines". Currently we still only use ``SQLAlchemy`` under the hood, but more engines are planned to be supported such as ``turbodbc`` (:issue:`36893`) +- Partially initialized :class:`CategoricalDtype` objects (i.e. those with ``categories=None``) will no longer compare as equal to fully initialized dtype objects. +- Added new ``engine`` and ``**engine_kwargs`` parameters to :meth:`DataFrame.to_sql` to support other future "SQL engines". Currently we still only use ``SQLAlchemy`` under the hood, but more engines are planned to be supported such as `turbodbc `_ (:issue:`36893`) Build -===== +^^^^^ - Documentation in ``.pptx`` and ``.pdf`` formats are no longer included in wheels or source distributions. (:issue:`30741`) @@ -657,29 +658,28 @@ Deprecations - Deprecated allowing scalars to be passed to the :class:`Categorical` constructor (:issue:`38433`) - Deprecated constructing :class:`CategoricalIndex` without passing list-like data (:issue:`38944`) - Deprecated allowing subclass-specific keyword arguments in the :class:`Index` constructor, use the specific subclass directly instead (:issue:`14093`, :issue:`21311`, :issue:`22315`, :issue:`26974`) -- Deprecated ``astype`` of datetimelike (``timedelta64[ns]``, ``datetime64[ns]``, ``Datetime64TZDtype``, ``PeriodDtype``) to integer dtypes, use ``values.view(...)`` instead (:issue:`38544`) +- Deprecated using the :meth:`astype` method of datetimelike (``timedelta64[ns]``, ``datetime64[ns]``, ``Datetime64TZDtype``, ``PeriodDtype``) to convert to integer dtypes, use ``values.view(...)`` instead (:issue:`38544`) - Deprecated :meth:`MultiIndex.is_lexsorted` and :meth:`MultiIndex.lexsort_depth`, use :meth:`MultiIndex.is_monotonic_increasing` instead (:issue:`32259`) - Deprecated keyword ``try_cast`` in :meth:`Series.where`, :meth:`Series.mask`, :meth:`DataFrame.where`, :meth:`DataFrame.mask`; cast results manually if desired (:issue:`38836`) -- Deprecated comparison of :class:`Timestamp` object with ``datetime.date`` objects. Instead of e.g. ``ts <= mydate`` use ``ts <= pd.Timestamp(mydate)`` or ``ts.date() <= mydate`` (:issue:`36131`) +- Deprecated comparison of :class:`Timestamp` objects with ``datetime.date`` objects. Instead of e.g. ``ts <= mydate`` use ``ts <= pd.Timestamp(mydate)`` or ``ts.date() <= mydate`` (:issue:`36131`) - Deprecated :attr:`Rolling.win_type` returning ``"freq"`` (:issue:`38963`) - Deprecated :attr:`Rolling.is_datetimelike` (:issue:`38963`) - Deprecated :class:`DataFrame` indexer for :meth:`Series.__setitem__` and :meth:`DataFrame.__setitem__` (:issue:`39004`) -- Deprecated :meth:`core.window.ewm.ExponentialMovingWindow.vol` (:issue:`39220`) +- Deprecated :meth:`ExponentialMovingWindow.vol` (:issue:`39220`) - Using ``.astype`` to convert between ``datetime64[ns]`` dtype and :class:`DatetimeTZDtype` is deprecated and will raise in a future version, use ``obj.tz_localize`` or ``obj.dt.tz_localize`` instead (:issue:`38622`) - Deprecated casting ``datetime.date`` objects to ``datetime64`` when used as ``fill_value`` in :meth:`DataFrame.unstack`, :meth:`DataFrame.shift`, :meth:`Series.shift`, and :meth:`DataFrame.reindex`, pass ``pd.Timestamp(dateobj)`` instead (:issue:`39767`) - Deprecated :meth:`.Styler.set_na_rep` and :meth:`.Styler.set_precision` in favour of :meth:`.Styler.format` with ``na_rep`` and ``precision`` as existing and new input arguments respectively (:issue:`40134`, :issue:`40425`) - Deprecated allowing partial failure in :meth:`Series.transform` and :meth:`DataFrame.transform` when ``func`` is list-like or dict-like and raises anything but ``TypeError``; ``func`` raising anything but a ``TypeError`` will raise in a future version (:issue:`40211`) -- Deprecated arguments ``error_bad_lines`` and ``warn_bad_lines`` in :meth:``read_csv`` and :meth:``read_table`` in favor of argument ``on_bad_lines`` (:issue:`15122`) +- Deprecated arguments ``error_bad_lines`` and ``warn_bad_lines`` in :meth:`read_csv` and :meth:`read_table` in favor of argument ``on_bad_lines`` (:issue:`15122`) - Deprecated support for ``np.ma.mrecords.MaskedRecords`` in the :class:`DataFrame` constructor, pass ``{name: data[name] for name in data.dtype.names}`` instead (:issue:`40363`) -- Deprecated using :func:`merge` or :func:`join` on a different number of levels (:issue:`34862`) +- Deprecated using :func:`merge`, :meth:`DataFrame.merge`, :meth:`DataFrame.join` on a different number of levels (:issue:`34862`) - Deprecated the use of ``**kwargs`` in :class:`.ExcelWriter`; use the keyword argument ``engine_kwargs`` instead (:issue:`40430`) -- Deprecated the ``level`` keyword for :class:`DataFrame` and :class:`Series` aggregations; use groupby instead (:issue:`39983`) -- The ``inplace`` parameter of :meth:`Categorical.remove_categories`, :meth:`Categorical.add_categories`, :meth:`Categorical.reorder_categories`, :meth:`Categorical.rename_categories`, :meth:`Categorical.set_categories` is deprecated and will be removed in a future version (:issue:`37643`) +- Deprecated the ``level`` keyword for :class:`DataFrame` and :class:`Series` aggregations (e.g. :meth:`DataFrame.sum`); use groupby instead (:issue:`39983`) +- Deprecated the ``inplace`` parameter of :meth:`Categorical.remove_categories`, :meth:`Categorical.add_categories`, :meth:`Categorical.reorder_categories`, :meth:`Categorical.rename_categories`, :meth:`Categorical.set_categories` and will be removed in a future version (:issue:`37643`) - Deprecated :func:`merge` producing duplicated columns through the ``suffixes`` keyword and already existing columns (:issue:`22818`) -- Deprecated setting :attr:`Categorical._codes`, create a new :class:`Categorical` with the desired codes instead (:issue:`40606`) - Deprecated the ``convert_float`` optional argument in :func:`read_excel` and :meth:`ExcelFile.parse` (:issue:`41127`) - Deprecated behavior of :meth:`DatetimeIndex.union` with mixed timezones; in a future version both will be cast to UTC instead of object dtype (:issue:`39328`) -- Deprecated using ``usecols`` with out of bounds indices for ``read_csv`` with ``engine="c"`` (:issue:`25623`) +- Deprecated using ``usecols`` with out of bounds indices for :func:`read_csv` with ``engine="c"`` (:issue:`25623`) - Deprecated passing arguments as positional (except for ``"codes"``) in :meth:`MultiIndex.codes` (:issue:`41485`) - Deprecated passing arguments as positional in :meth:`Index.set_names` and :meth:`MultiIndex.set_names` (except for ``names``) (:issue:`41485`) - Deprecated passing arguments (apart from ``cond`` and ``other``) as positional in :meth:`DataFrame.mask` and :meth:`Series.mask` (:issue:`41485`) @@ -692,10 +692,10 @@ Deprecations - Deprecated passing arguments as positional in :meth:`DataFrame.set_index` (other than ``"keys"``) (:issue:`41485`) - Deprecated passing arguments as positional (except for ``"levels"``) in :meth:`MultiIndex.set_levels` (:issue:`41485`) - Deprecated passing arguments as positional in :meth:`DataFrame.sort_index` and :meth:`Series.sort_index` (:issue:`41485`) -- Deprecated passing arguments as positional in :meth:`DataFrame.drop_duplicates` (except for ``subset``), :meth:`Series.drop_duplicates`, :meth:`Index.drop_duplicates` and :meth:`MultiIndex.drop_duplicates`(:issue:`41485`) +- Deprecated passing arguments as positional in :meth:`DataFrame.drop_duplicates` (except for ``subset``), :meth:`Series.drop_duplicates`, :meth:`Index.drop_duplicates` and :meth:`MultiIndex.drop_duplicates` (:issue:`41485`) - Deprecated passing arguments (apart from ``value``) as positional in :meth:`DataFrame.fillna` and :meth:`Series.fillna` (:issue:`41485`) - Deprecated passing arguments as positional in :meth:`DataFrame.reset_index` (other than ``"level"``) and :meth:`Series.reset_index` (:issue:`41485`) -- Deprecated construction of :class:`Series` or :class:`DataFrame` with ``DatetimeTZDtype`` data and ``datetime64[ns]`` dtype. Use ``Series(data).dt.tz_localize(None)`` instead (:issue:`41555`,:issue:`33401`) +- Deprecated construction of :class:`Series` or :class:`DataFrame` with ``DatetimeTZDtype`` data and ``datetime64[ns]`` dtype. Use ``Series(data).dt.tz_localize(None)`` instead (:issue:`41555`, :issue:`33401`) - Deprecated passing arguments as positional in :meth:`DataFrame.set_axis` and :meth:`Series.set_axis` (other than ``"labels"``) (:issue:`41485`) - Deprecated passing arguments as positional in :meth:`DataFrame.where` and :meth:`Series.where` (other than ``"cond"`` and ``"other"``) (:issue:`41485`) - Deprecated passing arguments as positional (other than ``filepath_or_buffer``) in :func:`read_csv` (:issue:`41485`) @@ -705,10 +705,10 @@ Deprecations .. _whatsnew_130.deprecations.nuisance_columns: -Deprecated Dropping Nuisance Columns in DataFrame Reductions and DataFrameGroupBy Operations +Deprecated dropping nuisance columns in DataFrame reductions and DataFrameGroupBy operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The default of calling a reduction (.min, .max, .sum, ...) on a :class:`DataFrame` with -``numeric_only=None`` (the default, columns on which the reduction raises ``TypeError`` +Calling a reduction (e.g. ``.min``, ``.max``, ``.sum``) on a :class:`DataFrame` with +``numeric_only=None`` (the default), columns where the reduction raises a ``TypeError`` are silently ignored and dropped from the result. This behavior is deprecated. In a future version, the ``TypeError`` will be raised, @@ -792,17 +792,17 @@ Performance improvements - Performance improvement in :meth:`IntervalIndex.isin` (:issue:`38353`) - Performance improvement in :meth:`Series.mean` for nullable data types (:issue:`34814`) - Performance improvement in :meth:`Series.isin` for nullable data types (:issue:`38340`) -- Performance improvement in :meth:`DataFrame.fillna` with ``method="pad|backfill"`` for nullable floating and nullable integer dtypes (:issue:`39953`) -- Performance improvement in :meth:`DataFrame.corr` for method=kendall (:issue:`28329`) -- Performance improvement in :meth:`core.window.rolling.Rolling.corr` and :meth:`core.window.rolling.Rolling.cov` (:issue:`39388`) -- Performance improvement in :meth:`core.window.rolling.RollingGroupby.corr`, :meth:`core.window.expanding.ExpandingGroupby.corr`, :meth:`core.window.expanding.ExpandingGroupby.corr` and :meth:`core.window.expanding.ExpandingGroupby.cov` (:issue:`39591`) +- Performance improvement in :meth:`DataFrame.fillna` with ``method="pad"`` or ``method="backfill"`` for nullable floating and nullable integer dtypes (:issue:`39953`) +- Performance improvement in :meth:`DataFrame.corr` for ``method=kendall`` (:issue:`28329`) +- Performance improvement in :meth:`.Rolling.corr` and :meth:`.Rolling.cov` (:issue:`39388`) +- Performance improvement in :meth:`.RollingGroupby.corr`, :meth:`.ExpandingGroupby.corr`, :meth:`.ExpandingGroupby.corr` and :meth:`.ExpandingGroupby.cov` (:issue:`39591`) - Performance improvement in :func:`unique` for object data type (:issue:`37615`) -- Performance improvement in :func:`pd.json_normalize` for basic cases (including separators) (:issue:`40035` :issue:`15621`) +- Performance improvement in :func:`json_normalize` for basic cases (including separators) (:issue:`40035` :issue:`15621`) - Performance improvement in :class:`core.window.rolling.ExpandingGroupby` aggregation methods (:issue:`39664`) - Performance improvement in :class:`Styler` where render times are more than 50% reduced (:issue:`39972` :issue:`39952`) -- Performance improvement in :meth:`core.window.ewm.ExponentialMovingWindow.mean` with ``times`` (:issue:`39784`) +- Performance improvement in :meth:`.ExponentialMovingWindow.mean` with ``times`` (:issue:`39784`) - Performance improvement in :meth:`.GroupBy.apply` when requiring the python fallback implementation (:issue:`40176`) -- Performance improvement in the conversion of pyarrow boolean array to a pandas nullable boolean array (:issue:`41051`) +- Performance improvement in the conversion of a PyArrow Boolean array to a pandas nullable Boolean array (:issue:`41051`) - Performance improvement for concatenation of data with type :class:`CategoricalDtype` (:issue:`40193`) - Performance improvement in :meth:`.GroupBy.cummin` and :meth:`.GroupBy.cummax` with nullable data types (:issue:`37493`) - Performance improvement in :meth:`Series.nunique` with nan values (:issue:`40865`) @@ -818,18 +818,17 @@ Bug fixes Categorical ^^^^^^^^^^^ - Bug in :class:`CategoricalIndex` incorrectly failing to raise ``TypeError`` when scalar data is passed (:issue:`38614`) -- Bug in ``CategoricalIndex.reindex`` failed when ``Index`` passed with elements all in category (:issue:`28690`) +- Bug in ``CategoricalIndex.reindex`` failed when the :class:`Index` passed was not categorical (:issue:`28690`) - Bug where constructing a :class:`Categorical` from an object-dtype array of ``date`` objects did not round-trip correctly with ``astype`` (:issue:`38552`) - Bug in constructing a :class:`DataFrame` from an ``ndarray`` and a :class:`CategoricalDtype` (:issue:`38857`) -- Bug in :meth:`DataFrame.reindex` was throwing ``IndexError`` when new index contained duplicates and old index was :class:`CategoricalIndex` (:issue:`38906`) - Bug in setting categorical values into an object-dtype column in a :class:`DataFrame` (:issue:`39136`) -- Bug in :meth:`DataFrame.reindex` was raising ``IndexError`` when new index contained duplicates and old index was :class:`CategoricalIndex` (:issue:`38906`) +- Bug in :meth:`DataFrame.reindex` was raising an ``IndexError`` when the new index contained duplicates and the old index was a :class:`CategoricalIndex` (:issue:`38906`) Datetimelike ^^^^^^^^^^^^ - Bug in :class:`DataFrame` and :class:`Series` constructors sometimes dropping nanoseconds from :class:`Timestamp` (resp. :class:`Timedelta`) ``data``, with ``dtype=datetime64[ns]`` (resp. ``timedelta64[ns]``) (:issue:`38032`) -- Bug in :meth:`DataFrame.first` and :meth:`Series.first` returning two months for offset one month when first day is last calendar day (:issue:`29623`) -- Bug in constructing a :class:`DataFrame` or :class:`Series` with mismatched ``datetime64`` data and ``timedelta64`` dtype, or vice-versa, failing to raise ``TypeError`` (:issue:`38575`, :issue:`38764`, :issue:`38792`) +- Bug in :meth:`DataFrame.first` and :meth:`Series.first` with an offset of one month returning an incorrect result when the first day is the last day of a month (:issue:`29623`) +- Bug in constructing a :class:`DataFrame` or :class:`Series` with mismatched ``datetime64`` data and ``timedelta64`` dtype, or vice-versa, failing to raise a ``TypeError`` (:issue:`38575`, :issue:`38764`, :issue:`38792`) - Bug in constructing a :class:`Series` or :class:`DataFrame` with a ``datetime`` object out of bounds for ``datetime64[ns]`` dtype or a ``timedelta`` object out of bounds for ``timedelta64[ns]`` dtype (:issue:`38792`, :issue:`38965`) - Bug in :meth:`DatetimeIndex.intersection`, :meth:`DatetimeIndex.symmetric_difference`, :meth:`PeriodIndex.intersection`, :meth:`PeriodIndex.symmetric_difference` always returning object-dtype when operating with :class:`CategoricalIndex` (:issue:`38741`) - Bug in :meth:`Series.where` incorrectly casting ``datetime64`` values to ``int64`` (:issue:`37682`) @@ -844,7 +843,7 @@ Timedelta ^^^^^^^^^ - Bug in constructing :class:`Timedelta` from ``np.timedelta64`` objects with non-nanosecond units that are out of bounds for ``timedelta64[ns]`` (:issue:`38965`) - Bug in constructing a :class:`TimedeltaIndex` incorrectly accepting ``np.datetime64("NaT")`` objects (:issue:`39462`) -- Bug in constructing :class:`Timedelta` from input string with only symbols and no digits failed to raise an error (:issue:`39710`) +- Bug in constructing :class:`Timedelta` from an input string with only symbols and no digits failed to raise an error (:issue:`39710`) - Bug in :class:`TimedeltaIndex` and :func:`to_timedelta` failing to raise when passed non-nanosecond ``timedelta64`` arrays that overflow when converting to ``timedelta64[ns]`` (:issue:`40008`) Timezones @@ -857,29 +856,29 @@ Numeric ^^^^^^^ - Bug in :meth:`DataFrame.quantile`, :meth:`DataFrame.sort_values` causing incorrect subsequent indexing behavior (:issue:`38351`) - Bug in :meth:`DataFrame.sort_values` raising an :class:`IndexError` for empty ``by`` (:issue:`40258`) -- Bug in :meth:`DataFrame.select_dtypes` with ``include=np.number`` now retains numeric ``ExtensionDtype`` columns (:issue:`35340`) +- Bug in :meth:`DataFrame.select_dtypes` with ``include=np.number`` would drop numeric ``ExtensionDtype`` columns (:issue:`35340`) - Bug in :meth:`DataFrame.mode` and :meth:`Series.mode` not keeping consistent integer :class:`Index` for empty input (:issue:`33321`) -- Bug in :meth:`DataFrame.rank` with ``np.inf`` and mixture of ``np.nan`` and ``np.inf`` (:issue:`32593`) -- Bug in :meth:`DataFrame.rank` with ``axis=0`` and columns holding incomparable types raising ``IndexError`` (:issue:`38932`) -- Bug in ``rank`` method for :class:`Series`, :class:`DataFrame`, :class:`DataFrameGroupBy`, and :class:`SeriesGroupBy` treating the most negative ``int64`` value as missing (:issue:`32859`) -- Bug in :func:`select_dtypes` different behavior between Windows and Linux with ``include="int"`` (:issue:`36569`) -- Bug in :meth:`DataFrame.apply` and :meth:`DataFrame.agg` when passed argument ``func="size"`` would operate on the entire ``DataFrame`` instead of rows or columns (:issue:`39934`) -- Bug in :meth:`DataFrame.transform` would raise ``SpecificationError`` when passed a dictionary and columns were missing; will now raise a ``KeyError`` instead (:issue:`40004`) -- Bug in :meth:`DataFrameGroupBy.rank` giving incorrect results with ``pct=True`` and equal values between consecutive groups (:issue:`40518`) +- Bug in :meth:`DataFrame.rank` when the DataFrame contained ``np.inf`` (:issue:`32593`) +- Bug in :meth:`DataFrame.rank` with ``axis=0`` and columns holding incomparable types raising an ``IndexError`` (:issue:`38932`) +- Bug in :meth:`Series.rank`, :meth:`DataFrame.rank`, and :meth:`.GroupBy.rank` treating the most negative ``int64`` value as missing (:issue:`32859`) +- Bug in :meth:`DataFrame.select_dtypes` different behavior between Windows and Linux with ``include="int"`` (:issue:`36596`) +- Bug in :meth:`DataFrame.apply` and :meth:`DataFrame.agg` when passed the argument ``func="size"`` would operate on the entire ``DataFrame`` instead of rows or columns (:issue:`39934`) +- Bug in :meth:`DataFrame.transform` would raise a ``SpecificationError`` when passed a dictionary and columns were missing; will now raise a ``KeyError`` instead (:issue:`40004`) +- Bug in :meth:`.GroupBy.rank` giving incorrect results with ``pct=True`` and equal values between consecutive groups (:issue:`40518`) - Bug in :meth:`Series.count` would result in an ``int32`` result on 32-bit platforms when argument ``level=None`` (:issue:`40908`) -- Bug in :class:`Series` and :class:`DataFrame` reductions with methods ``any`` and ``all`` not returning boolean results for object data (:issue:`12863`, :issue:`35450`, :issue:`27709`) -- Bug in :meth:`Series.clip` would fail if series contains NA values and has nullable int or float as a data type (:issue:`40851`) +- Bug in :class:`Series` and :class:`DataFrame` reductions with methods ``any`` and ``all`` not returning Boolean results for object data (:issue:`12863`, :issue:`35450`, :issue:`27709`) +- Bug in :meth:`Series.clip` would fail if the Series contains NA values and has nullable int or float as a data type (:issue:`40851`) Conversion ^^^^^^^^^^ -- Bug in :meth:`Series.to_dict` with ``orient='records'`` now returns python native types (:issue:`25969`) +- Bug in :meth:`Series.to_dict` with ``orient='records'`` now returns Python native types (:issue:`25969`) - Bug in :meth:`Series.view` and :meth:`Index.view` when converting between datetime-like (``datetime64[ns]``, ``datetime64[ns, tz]``, ``timedelta64``, ``period``) dtypes (:issue:`39788`) - Bug in creating a :class:`DataFrame` from an empty ``np.recarray`` not retaining the original dtypes (:issue:`40121`) -- Bug in :class:`DataFrame` failing to raise ``TypeError`` when constructing from a ``frozenset`` (:issue:`40163`) +- Bug in :class:`DataFrame` failing to raise a ``TypeError`` when constructing from a ``frozenset`` (:issue:`40163`) - Bug in :class:`Index` construction silently ignoring a passed ``dtype`` when the data cannot be cast to that dtype (:issue:`21311`) -- Bug in :meth:`StringArray.astype` falling back to numpy and raising when converting to ``dtype='categorical'`` (:issue:`40450`) -- Bug in :func:`factorize` where, when given an array with a numeric numpy dtype lower than int64, uint64 and float64, the unique values did not keep their original dtype (:issue:`41132`) -- Bug in :class:`DataFrame` construction with a dictionary containing an arraylike with ``ExtensionDtype`` and ``copy=True`` failing to make a copy (:issue:`38939`) +- Bug in :meth:`StringArray.astype` falling back to NumPy and raising when converting to ``dtype='categorical'`` (:issue:`40450`) +- Bug in :func:`factorize` where, when given an array with a numeric NumPy dtype lower than int64, uint64 and float64, the unique values did not keep their original dtype (:issue:`41132`) +- Bug in :class:`DataFrame` construction with a dictionary containing an array-like with ``ExtensionDtype`` and ``copy=True`` failing to make a copy (:issue:`38939`) - Bug in :meth:`qcut` raising error when taking ``Float64DType`` as input (:issue:`40730`) - Bug in :class:`DataFrame` and :class:`Series` construction with ``datetime64[ns]`` data and ``dtype=object`` resulting in ``datetime`` objects instead of :class:`Timestamp` objects (:issue:`41599`) - Bug in :class:`DataFrame` and :class:`Series` construction with ``timedelta64[ns]`` data and ``dtype=object`` resulting in ``np.timedelta64`` objects instead of :class:`Timedelta` objects (:issue:`41599`) @@ -889,110 +888,111 @@ Strings - Bug in the conversion from ``pyarrow.ChunkedArray`` to :class:`~arrays.StringArray` when the original had zero chunks (:issue:`41040`) - Bug in :meth:`Series.replace` and :meth:`DataFrame.replace` ignoring replacements with ``regex=True`` for ``StringDType`` data (:issue:`41333`, :issue:`35977`) -- Bug in :meth:`Series.str.extract` with :class:`~arrays.StringArray` returning object dtype for empty :class:`DataFrame` (:issue:`41441`) +- Bug in :meth:`Series.str.extract` with :class:`~arrays.StringArray` returning object dtype for an empty :class:`DataFrame` (:issue:`41441`) - Bug in :meth:`Series.str.replace` where the ``case`` argument was ignored when ``regex=False`` (:issue:`41602`) Interval ^^^^^^^^ - Bug in :meth:`IntervalIndex.intersection` and :meth:`IntervalIndex.symmetric_difference` always returning object-dtype when operating with :class:`CategoricalIndex` (:issue:`38653`, :issue:`38741`) -- Bug in :meth:`IntervalIndex.intersection` returning duplicates when at least one of both Indexes has duplicates which are present in the other (:issue:`38743`) -- :meth:`IntervalIndex.union`, :meth:`IntervalIndex.intersection`, :meth:`IntervalIndex.difference`, and :meth:`IntervalIndex.symmetric_difference` now cast to the appropriate dtype instead of raising ``TypeError`` when operating with another :class:`IntervalIndex` with incompatible dtype (:issue:`39267`) -- :meth:`PeriodIndex.union`, :meth:`PeriodIndex.intersection`, :meth:`PeriodIndex.symmetric_difference`, :meth:`PeriodIndex.difference` now cast to object dtype instead of raising ``IncompatibleFrequency`` when operating with another :class:`PeriodIndex` with incompatible dtype (:issue:`??`) +- Bug in :meth:`IntervalIndex.intersection` returning duplicates when at least one of the :class:`Index` objects have duplicates which are present in the other (:issue:`38743`) +- :meth:`IntervalIndex.union`, :meth:`IntervalIndex.intersection`, :meth:`IntervalIndex.difference`, and :meth:`IntervalIndex.symmetric_difference` now cast to the appropriate dtype instead of raising a ``TypeError`` when operating with another :class:`IntervalIndex` with incompatible dtype (:issue:`39267`) +- :meth:`PeriodIndex.union`, :meth:`PeriodIndex.intersection`, :meth:`PeriodIndex.symmetric_difference`, :meth:`PeriodIndex.difference` now cast to object dtype instead of raising ``IncompatibleFrequency`` when operating with another :class:`PeriodIndex` with incompatible dtype (:issue:`39306`) Indexing ^^^^^^^^ - Bug in :meth:`Index.union` and :meth:`MultiIndex.union` dropping duplicate ``Index`` values when ``Index`` was not monotonic or ``sort`` was set to ``False`` (:issue:`36289`, :issue:`31326`, :issue:`40862`) - Bug in :meth:`CategoricalIndex.get_indexer` failing to raise ``InvalidIndexError`` when non-unique (:issue:`38372`) -- Bug in :meth:`Series.loc` raising ``ValueError`` when input was filtered with a boolean list and values to set were a list with lower dimension (:issue:`20438`) +- Bug in :meth:`Series.loc` raising a ``ValueError`` when input was filtered with a Boolean list and values to set were a list with lower dimension (:issue:`20438`) - Bug in inserting many new columns into a :class:`DataFrame` causing incorrect subsequent indexing behavior (:issue:`38380`) -- Bug in :meth:`DataFrame.__setitem__` raising ``ValueError`` when setting multiple values to duplicate columns (:issue:`15695`) +- Bug in :meth:`DataFrame.__setitem__` raising a ``ValueError`` when setting multiple values to duplicate columns (:issue:`15695`) - Bug in :meth:`DataFrame.loc`, :meth:`Series.loc`, :meth:`DataFrame.__getitem__` and :meth:`Series.__getitem__` returning incorrect elements for non-monotonic :class:`DatetimeIndex` for string slices (:issue:`33146`) -- Bug in :meth:`DataFrame.reindex` and :meth:`Series.reindex` with timezone aware indexes raising ``TypeError`` for ``method="ffill"`` and ``method="bfill"`` and specified ``tolerance`` (:issue:`38566`) +- Bug in :meth:`DataFrame.reindex` and :meth:`Series.reindex` with timezone aware indexes raising a ``TypeError`` for ``method="ffill"`` and ``method="bfill"`` and specified ``tolerance`` (:issue:`38566`) - Bug in :meth:`DataFrame.reindex` with ``datetime64[ns]`` or ``timedelta64[ns]`` incorrectly casting to integers when the ``fill_value`` requires casting to object dtype (:issue:`39755`) -- Bug in :meth:`DataFrame.__setitem__` raising ``ValueError`` with empty :class:`DataFrame` and specified columns for string indexer and non empty :class:`DataFrame` to set (:issue:`38831`) -- Bug in :meth:`DataFrame.loc.__setitem__` raising ValueError when expanding unique column for :class:`DataFrame` with duplicate columns (:issue:`38521`) +- Bug in :meth:`DataFrame.__setitem__` raising a ``ValueError`` when setting on an empty :class:`DataFrame` using specified columns and a nonempty :class:`DataFrame` value (:issue:`38831`) +- Bug in :meth:`DataFrame.loc.__setitem__` raising ValueError when operating on a unique column when the :class:`DataFrame` has duplicate columns (:issue:`38521`) - Bug in :meth:`DataFrame.iloc.__setitem__` and :meth:`DataFrame.loc.__setitem__` with mixed dtypes when setting with a dictionary value (:issue:`38335`) -- Bug in :meth:`Series.loc.__setitem__` and :meth:`DataFrame.loc.__setitem__` raising ``KeyError`` for boolean Iterator indexer (:issue:`39614`) -- Bug in :meth:`Series.iloc` and :meth:`DataFrame.iloc` raising ``KeyError`` for Iterator indexer (:issue:`39614`) -- Bug in :meth:`DataFrame.__setitem__` not raising ``ValueError`` when right hand side is a :class:`DataFrame` with wrong number of columns (:issue:`38604`) -- Bug in :meth:`Series.__setitem__` raising ``ValueError`` when setting a :class:`Series` with a scalar indexer (:issue:`38303`) -- Bug in :meth:`DataFrame.loc` dropping levels of :class:`MultiIndex` when :class:`DataFrame` used as input has only one row (:issue:`10521`) -- Bug in :meth:`DataFrame.__getitem__` and :meth:`Series.__getitem__` always raising ``KeyError`` when slicing with existing strings an :class:`Index` with milliseconds (:issue:`33589`) -- Bug in setting ``timedelta64`` or ``datetime64`` values into numeric :class:`Series` failing to cast to object dtype (:issue:`39086`, issue:`39619`) +- Bug in :meth:`Series.loc.__setitem__` and :meth:`DataFrame.loc.__setitem__` raising ``KeyError`` for a Boolean generator (:issue:`39614`) +- Bug in :meth:`Series.iloc` and :meth:`DataFrame.iloc` raising a ``KeyError`` for iterator indexer (:issue:`39614`) +- Bug in :meth:`DataFrame.__setitem__` not raising a ``ValueError`` when the right hand side is a :class:`DataFrame` with wrong number of columns (:issue:`38604`) +- Bug in :meth:`Series.__setitem__` raising a ``ValueError`` when setting a :class:`Series` with a scalar indexer (:issue:`38303`) +- Bug in :meth:`DataFrame.loc` dropping levels of a :class:`MultiIndex` when the :class:`DataFrame` used as input has only one row (:issue:`10521`) +- Bug in :meth:`DataFrame.__getitem__` and :meth:`Series.__getitem__` always raising ``KeyError`` when slicing with existing strings where the :class:`Index` has milliseconds (:issue:`33589`) +- Bug in setting ``timedelta64`` or ``datetime64`` values into numeric :class:`Series` failing to cast to object dtype (:issue:`39086`, :issue:`39619`) - Bug in setting :class:`Interval` values into a :class:`Series` or :class:`DataFrame` with mismatched :class:`IntervalDtype` incorrectly casting the new values to the existing dtype (:issue:`39120`) -- Bug in setting ``datetime64`` values into a :class:`Series` with integer-dtype incorrect casting the datetime64 values to integers (:issue:`39266`) +- Bug in setting ``datetime64`` values into a :class:`Series` with integer-dtype incorrectly casting the datetime64 values to integers (:issue:`39266`) - Bug in setting ``np.datetime64("NaT")`` into a :class:`Series` with :class:`Datetime64TZDtype` incorrectly treating the timezone-naive value as timezone-aware (:issue:`39769`) -- Bug in :meth:`Index.get_loc` not raising ``KeyError`` when method is specified for ``NaN`` value when ``NaN`` is not in :class:`Index` (:issue:`39382`) +- Bug in :meth:`Index.get_loc` not raising ``KeyError`` when ``key=NaN`` and ``method`` is specified but ``NaN`` is not in the :class:`Index` (:issue:`39382`) - Bug in :meth:`DatetimeIndex.insert` when inserting ``np.datetime64("NaT")`` into a timezone-aware index incorrectly treating the timezone-naive value as timezone-aware (:issue:`39769`) - Bug in incorrectly raising in :meth:`Index.insert`, when setting a new column that cannot be held in the existing ``frame.columns``, or in :meth:`Series.reset_index` or :meth:`DataFrame.reset_index` instead of casting to a compatible dtype (:issue:`39068`) - Bug in :meth:`RangeIndex.append` where a single object of length 1 was concatenated incorrectly (:issue:`39401`) - Bug in :meth:`RangeIndex.astype` where when converting to :class:`CategoricalIndex`, the categories became a :class:`Int64Index` instead of a :class:`RangeIndex` (:issue:`41263`) -- Bug in setting ``numpy.timedelta64`` values into an object-dtype :class:`Series` using a boolean indexer (:issue:`39488`) +- Bug in setting ``numpy.timedelta64`` values into an object-dtype :class:`Series` using a Boolean indexer (:issue:`39488`) - Bug in setting numeric values into a into a boolean-dtypes :class:`Series` using ``at`` or ``iat`` failing to cast to object-dtype (:issue:`39582`) - Bug in :meth:`DataFrame.__setitem__` and :meth:`DataFrame.iloc.__setitem__` raising ``ValueError`` when trying to index with a row-slice and setting a list as values (:issue:`40440`) -- Bug in :meth:`DataFrame.loc` not raising ``KeyError`` when key was not found in :class:`MultiIndex` when levels contain more values than used (:issue:`41170`) -- Bug in :meth:`DataFrame.loc.__setitem__` when setting-with-expansion incorrectly raising when the index in the expanding axis contains duplicates (:issue:`40096`) -- Bug in :meth:`DataFrame.loc.__getitem__` with :class:`MultiIndex` casting to float when at least one column is from has float dtype and we retrieve a scalar (:issue:`41369`) -- Bug in :meth:`DataFrame.loc` incorrectly matching non-boolean index elements (:issue:`20432`) +- Bug in :meth:`DataFrame.loc` not raising ``KeyError`` when the key was not found in :class:`MultiIndex` and the levels were not fully specified (:issue:`41170`) +- Bug in :meth:`DataFrame.loc.__setitem__` when setting-with-expansion incorrectly raising when the index in the expanding axis contained duplicates (:issue:`40096`) +- Bug in :meth:`DataFrame.loc.__getitem__` with :class:`MultiIndex` casting to float when at least one index column has float dtype and we retrieve a scalar (:issue:`41369`) +- Bug in :meth:`DataFrame.loc` incorrectly matching non-Boolean index elements (:issue:`20432`) - Bug in :meth:`Series.__delitem__` with ``ExtensionDtype`` incorrectly casting to ``ndarray`` (:issue:`40386`) -- Bug in :meth:`DataFrame.loc` returning :class:`MultiIndex` in wrong order if indexer has duplicates (:issue:`40978`) -- Bug in :meth:`DataFrame.__setitem__` raising ``TypeError`` when using a str subclass as the column name with a :class:`DatetimeIndex` (:issue:`37366`) -- Bug in :meth:`PeriodIndex.get_loc` failing to raise ``KeyError`` when given a :class:`Period` with a mismatched ``freq`` (:issue:`41670`) +- Bug in :meth:`DataFrame.loc` returning a :class:`MultiIndex` in the wrong order if an indexer has duplicates (:issue:`40978`) +- Bug in :meth:`DataFrame.__setitem__` raising a ``TypeError`` when using a ``str`` subclass as the column name with a :class:`DatetimeIndex` (:issue:`37366`) +- Bug in :meth:`PeriodIndex.get_loc` failing to raise a ``KeyError`` when given a :class:`Period` with a mismatched ``freq`` (:issue:`41670`) Missing ^^^^^^^ -- Bug in :class:`Grouper` now correctly propagates ``dropna`` argument and :meth:`DataFrameGroupBy.transform` now correctly handles missing values for ``dropna=True`` (:issue:`35612`) -- Bug in :func:`isna`, and :meth:`Series.isna`, :meth:`Index.isna`, :meth:`DataFrame.isna` (and the corresponding ``notna`` functions) not recognizing ``Decimal("NaN")`` objects (:issue:`39409`) -- Bug in :meth:`DataFrame.fillna` not accepting dictionary for ``downcast`` keyword (:issue:`40809`) +- Bug in :class:`Grouper` did not correctly propagate the ``dropna`` argument; :meth:`.DataFrameGroupBy.transform` now correctly handles missing values for ``dropna=True`` (:issue:`35612`) +- Bug in :func:`isna`, and :meth:`Series.isna`, :meth:`Index.isna`, :meth:`DataFrame.isna`, and the corresponding ``notna`` functions not recognizing ``Decimal("NaN")`` objects (:issue:`39409`) +- Bug in :meth:`DataFrame.fillna` not accepting a dictionary for the ``downcast`` keyword (:issue:`40809`) - Bug in :func:`isna` not returning a copy of the mask for nullable types, causing any subsequent mask modification to change the original array (:issue:`40935`) - Bug in :class:`DataFrame` construction with float data containing ``NaN`` and an integer ``dtype`` casting instead of retaining the ``NaN`` (:issue:`26919`) MultiIndex ^^^^^^^^^^ -- Bug in :meth:`DataFrame.drop` raising ``TypeError`` when :class:`MultiIndex` is non-unique and ``level`` is not provided (:issue:`36293`) -- Bug in :meth:`MultiIndex.intersection` duplicating ``NaN`` in result (:issue:`38623`) -- Bug in :meth:`MultiIndex.equals` incorrectly returning ``True`` when :class:`MultiIndex` containing ``NaN`` even when they are differently ordered (:issue:`38439`) -- Bug in :meth:`MultiIndex.intersection` always returning empty when intersecting with :class:`CategoricalIndex` (:issue:`38653`) -- Bug in :meth:`MultiIndex.reindex` raising ``ValueError`` with empty MultiIndex and indexing only a specific level (:issue:`41170`) +- Bug in :meth:`DataFrame.drop` raising a ``TypeError`` when the :class:`MultiIndex` is non-unique and ``level`` is not provided (:issue:`36293`) +- Bug in :meth:`MultiIndex.intersection` duplicating ``NaN`` in the result (:issue:`38623`) +- Bug in :meth:`MultiIndex.equals` incorrectly returning ``True`` when the :class:`MultiIndex` contained ``NaN`` even when they are differently ordered (:issue:`38439`) +- Bug in :meth:`MultiIndex.intersection` always returning an empty result when intersecting with :class:`CategoricalIndex` (:issue:`38653`) +- Bug in :meth:`MultiIndex.reindex` raising a ``ValueError`` when used on an empty :class:`MultiIndex` and indexing only a specific level (:issue:`41170`) I/O ^^^ - Bug in :meth:`Index.__repr__` when ``display.max_seq_items=1`` (:issue:`38415`) -- Bug in :func:`read_csv` not recognizing scientific notation if decimal is set for ``engine="python"`` (:issue:`31920`) +- Bug in :func:`read_csv` not recognizing scientific notation if the argument ``decimal`` is set and ``engine="python"`` (:issue:`31920`) - Bug in :func:`read_csv` interpreting ``NA`` value as comment, when ``NA`` does contain the comment string fixed for ``engine="python"`` (:issue:`34002`) -- Bug in :func:`read_csv` raising ``IndexError`` with multiple header columns and ``index_col`` specified when file has no data rows (:issue:`38292`) -- Bug in :func:`read_csv` not accepting ``usecols`` with different length than ``names`` for ``engine="python"`` (:issue:`16469`) +- Bug in :func:`read_csv` raising an ``IndexError`` with multiple header columns and ``index_col`` is specified when the file has no data rows (:issue:`38292`) +- Bug in :func:`read_csv` not accepting ``usecols`` with a different length than ``names`` for ``engine="python"`` (:issue:`16469`) - Bug in :meth:`read_csv` returning object dtype when ``delimiter=","`` with ``usecols`` and ``parse_dates`` specified for ``engine="python"`` (:issue:`35873`) -- Bug in :func:`read_csv` raising ``TypeError`` when ``names`` and ``parse_dates`` is specified for ``engine="c"`` (:issue:`33699`) -- Bug in :func:`read_clipboard`, :func:`DataFrame.to_clipboard` not working in WSL (:issue:`38527`) -- Allow custom error values for parse_dates argument of :func:`read_sql`, :func:`read_sql_query` and :func:`read_sql_table` (:issue:`35185`) -- Bug in :func:`to_hdf` raising ``KeyError`` when trying to apply for subclasses of ``DataFrame`` or ``Series`` (:issue:`33748`) -- Bug in :meth:`~HDFStore.put` raising a wrong ``TypeError`` when saving a DataFrame with non-string dtype (:issue:`34274`) -- Bug in :func:`json_normalize` resulting in the first element of a generator object not being included in the returned ``DataFrame`` (:issue:`35923`) -- Bug in :func:`read_csv` applying thousands separator to date columns when column should be parsed for dates and ``usecols`` is specified for ``engine="python"`` (:issue:`39365`) -- Bug in :func:`read_excel` forward filling :class:`MultiIndex` names with multiple header and index columns specified (:issue:`34673`) -- :func:`read_excel` now respects :func:`set_option` (:issue:`34252`) -- Bug in :func:`read_csv` not switching ``true_values`` and ``false_values`` for nullable ``boolean`` dtype (:issue:`34655`) -- Bug in :func:`read_json` when ``orient="split"`` does not maintain numeric string index (:issue:`28556`) -- :meth:`read_sql` returned an empty generator if ``chunksize`` was no-zero and the query returned no results. Now returns a generator with a single empty dataframe (:issue:`34411`) -- Bug in :func:`read_hdf` returning unexpected records when filtering on categorical string columns using ``where`` parameter (:issue:`39189`) -- Bug in :func:`read_sas` raising ``ValueError`` when ``datetimes`` were null (:issue:`39725`) +- Bug in :func:`read_csv` raising a ``TypeError`` when ``names`` and ``parse_dates`` is specified for ``engine="c"`` (:issue:`33699`) +- Bug in :func:`read_clipboard` and :func:`DataFrame.to_clipboard` not working in WSL (:issue:`38527`) +- Allow custom error values for the ``parse_dates`` argument of :func:`read_sql`, :func:`read_sql_query` and :func:`read_sql_table` (:issue:`35185`) +- Bug in :meth:`DataFrame.to_hdf` and :meth:`Series.to_hdf` raising a ``KeyError`` when trying to apply for subclasses of ``DataFrame`` or ``Series`` (:issue:`33748`) +- Bug in :meth:`.HDFStore.put` raising a wrong ``TypeError`` when saving a DataFrame with non-string dtype (:issue:`34274`) +- Bug in :func:`json_normalize` resulting in the first element of a generator object not being included in the returned DataFrame (:issue:`35923`) +- Bug in :func:`read_csv` applying the thousands separator to date columns when the column should be parsed for dates and ``usecols`` is specified for ``engine="python"`` (:issue:`39365`) +- Bug in :func:`read_excel` forward filling :class:`MultiIndex` names when multiple header and index columns are specified (:issue:`34673`) +- Bug in :func:`read_excel` not respecting :func:`set_option` (:issue:`34252`) +- Bug in :func:`read_csv` not switching ``true_values`` and ``false_values`` for nullable Boolean dtype (:issue:`34655`) +- Bug in :func:`read_json` when ``orient="split"`` not maintaining a numeric string index (:issue:`28556`) +- :meth:`read_sql` returned an empty generator if ``chunksize`` was non-zero and the query returned no results. Now returns a generator with a single empty DataFrame (:issue:`34411`) +- Bug in :func:`read_hdf` returning unexpected records when filtering on categorical string columns using the ``where`` parameter (:issue:`39189`) +- Bug in :func:`read_sas` raising a ``ValueError`` when ``datetimes`` were null (:issue:`39725`) - Bug in :func:`read_excel` dropping empty values from single-column spreadsheets (:issue:`39808`) - Bug in :func:`read_excel` loading trailing empty rows/columns for some filetypes (:issue:`41167`) -- Bug in :func:`read_excel` raising ``AttributeError`` with ``MultiIndex`` header followed by two empty rows and no index, and bug affecting :func:`read_excel`, :func:`read_csv`, :func:`read_table`, :func:`read_fwf`, and :func:`read_clipboard` where one blank row after a ``MultiIndex`` header with no index would be dropped (:issue:`40442`) +- Bug in :func:`read_excel` raising an ``AttributeError`` when the excel file had a ``MultiIndex`` header followed by two empty rows and no index (:issue:`40442`) +- Bug in :func:`read_excel`, :func:`read_csv`, :func:`read_table`, :func:`read_fwf`, and :func:`read_clipboard` where one blank row after a ``MultiIndex`` header with no index would be dropped (:issue:`40442`) - Bug in :meth:`DataFrame.to_string` misplacing the truncation column when ``index=False`` (:issue:`40904`) - Bug in :meth:`DataFrame.to_string` adding an extra dot and misaligning the truncation row when ``index=False`` (:issue:`40904`) -- Bug in :func:`read_orc` always raising ``AttributeError`` (:issue:`40918`) -- Bug in :func:`read_csv` and :func:`read_table` silently ignoring ``prefix`` if ``names`` and ``prefix`` are defined, now raising ``ValueError`` (:issue:`39123`) -- Bug in :func:`read_csv` and :func:`read_excel` not respecting dtype for duplicated column name when ``mangle_dupe_cols`` is set to ``True`` (:issue:`35211`) -- Bug in :func:`read_csv` silently ignoring ``sep`` if ``delimiter`` and ``sep`` are defined, now raising ``ValueError`` (:issue:`39823`) +- Bug in :func:`read_orc` always raising an ``AttributeError`` (:issue:`40918`) +- Bug in :func:`read_csv` and :func:`read_table` silently ignoring ``prefix`` if ``names`` and ``prefix`` are defined, now raising a ``ValueError`` (:issue:`39123`) +- Bug in :func:`read_csv` and :func:`read_excel` not respecting the dtype for a duplicated column name when ``mangle_dupe_cols`` is set to ``True`` (:issue:`35211`) +- Bug in :func:`read_csv` silently ignoring ``sep`` if ``delimiter`` and ``sep`` are defined, now raising a ``ValueError`` (:issue:`39823`) - Bug in :func:`read_csv` and :func:`read_table` misinterpreting arguments when ``sys.setprofile`` had been previously called (:issue:`41069`) -- Bug in the conversion from pyarrow to pandas (e.g. for reading Parquet) with nullable dtypes and a pyarrow array whose data buffer size is not a multiple of dtype size (:issue:`40896`) -- Bug in :func:`read_excel` would raise an error when pandas could not determine the file type, even when user specified the ``engine`` argument (:issue:`41225`) +- Bug in the conversion from PyArrow to pandas (e.g. for reading Parquet) with nullable dtypes and a PyArrow array whose data buffer size is not a multiple of the dtype size (:issue:`40896`) +- Bug in :func:`read_excel` would raise an error when pandas could not determine the file type even though the user specified the ``engine`` argument (:issue:`41225`) - Bug in :func:`read_clipboard` copying from an excel file shifts values into the wrong column if there are null values in first column (:issue:`41108`) Period @@ -1004,123 +1004,123 @@ Period Plotting ^^^^^^^^ -- Bug in :func:`scatter_matrix` raising when 2d ``ax`` argument passed (:issue:`16253`) -- Prevent warnings when matplotlib's ``constrained_layout`` is enabled (:issue:`25261`) -- Bug in :func:`DataFrame.plot` was showing the wrong colors in the legend if the function was called repeatedly and some calls used ``yerr`` while others didn't (partial fix of :issue:`39522`) +- Bug in :func:`plotting.scatter_matrix` raising when 2d ``ax`` argument passed (:issue:`16253`) +- Prevent warnings when Matplotlib's ``constrained_layout`` is enabled (:issue:`25261`) +- Bug in :func:`DataFrame.plot` was showing the wrong colors in the legend if the function was called repeatedly and some calls used ``yerr`` while others didn't (:issue:`39522`) - Bug in :func:`DataFrame.plot` was showing the wrong colors in the legend if the function was called repeatedly and some calls used ``secondary_y`` and others use ``legend=False`` (:issue:`40044`) -- Bug in :meth:`DataFrame.plot.box` in box plot when ``dark_background`` theme was selected, caps or min/max markers for the plot was not visible (:issue:`40769`) +- Bug in :meth:`DataFrame.plot.box` when ``dark_background`` theme was selected, caps or min/max markers for the plot were not visible (:issue:`40769`) Groupby/resample/rolling ^^^^^^^^^^^^^^^^^^^^^^^^ -- Bug in :meth:`DataFrameGroupBy.agg` and :meth:`SeriesGroupBy.agg` with :class:`PeriodDtype` columns incorrectly casting results too aggressively (:issue:`38254`) -- Bug in :meth:`SeriesGroupBy.value_counts` where unobserved categories in a grouped categorical series were not tallied (:issue:`38672`) -- Bug in :meth:`SeriesGroupBy.value_counts` where error was raised on an empty series (:issue:`39172`) +- Bug in :meth:`.GroupBy.agg` with :class:`PeriodDtype` columns incorrectly casting results too aggressively (:issue:`38254`) +- Bug in :meth:`.SeriesGroupBy.value_counts` where unobserved categories in a grouped categorical Series were not tallied (:issue:`38672`) +- Bug in :meth:`.SeriesGroupBy.value_counts` where an error was raised on an empty Series (:issue:`39172`) - Bug in :meth:`.GroupBy.indices` would contain non-existent indices when null values were present in the groupby keys (:issue:`9304`) -- Fixed bug in :meth:`DataFrameGroupBy.sum` and :meth:`SeriesGroupBy.sum` causing loss of precision through using Kahan summation (:issue:`38778`) -- Fixed bug in :meth:`DataFrameGroupBy.cumsum`, :meth:`SeriesGroupBy.cumsum`, :meth:`DataFrameGroupBy.mean` and :meth:`SeriesGroupBy.mean` causing loss of precision through using Kahan summation (:issue:`38934`) -- Bug in :meth:`.Resampler.aggregate` and :meth:`DataFrame.transform` raising ``TypeError`` instead of ``SpecificationError`` when missing keys had mixed dtypes (:issue:`39025`) +- Fixed bug in :meth:`.GroupBy.sum` causing a loss of precision by now using Kahan summation (:issue:`38778`) +- Fixed bug in :meth:`.GroupBy.cumsum` and :meth:`.GroupBy.mean` causing loss of precision through using Kahan summation (:issue:`38934`) +- Bug in :meth:`.Resampler.aggregate` and :meth:`DataFrame.transform` raising a ``TypeError`` instead of ``SpecificationError`` when missing keys had mixed dtypes (:issue:`39025`) - Bug in :meth:`.DataFrameGroupBy.idxmin` and :meth:`.DataFrameGroupBy.idxmax` with ``ExtensionDtype`` columns (:issue:`38733`) - Bug in :meth:`Series.resample` would raise when the index was a :class:`PeriodIndex` consisting of ``NaT`` (:issue:`39227`) -- Bug in :meth:`core.window.rolling.RollingGroupby.corr` and :meth:`core.window.expanding.ExpandingGroupby.corr` where the groupby column would return 0 instead of ``np.nan`` when providing ``other`` that was longer than each group (:issue:`39591`) -- Bug in :meth:`core.window.expanding.ExpandingGroupby.corr` and :meth:`core.window.expanding.ExpandingGroupby.cov` where 1 would be returned instead of ``np.nan`` when providing ``other`` that was longer than each group (:issue:`39591`) +- Bug in :meth:`.RollingGroupby.corr` and :meth:`.ExpandingGroupby.corr` where the groupby column would return ``0`` instead of ``np.nan`` when providing ``other`` that was longer than each group (:issue:`39591`) +- Bug in :meth:`.ExpandingGroupby.corr` and :meth:`.ExpandingGroupby.cov` where ``1`` would be returned instead of ``np.nan`` when providing ``other`` that was longer than each group (:issue:`39591`) - Bug in :meth:`.GroupBy.mean`, :meth:`.GroupBy.median` and :meth:`DataFrame.pivot_table` not propagating metadata (:issue:`28283`) - Bug in :meth:`Series.rolling` and :meth:`DataFrame.rolling` not calculating window bounds correctly when window is an offset and dates are in descending order (:issue:`40002`) -- Bug in :class:`SeriesGroupBy` and :class:`DataFrameGroupBy` on an empty ``Series`` or ``DataFrame`` would lose index, columns, and/or data types when directly using the methods ``idxmax``, ``idxmin``, ``mad``, ``min``, ``max``, ``sum``, ``prod``, and ``skew`` or using them through ``apply``, ``aggregate``, or ``resample`` (:issue:`26411`) -- Bug in :meth:`DataFrameGroupBy.apply` where a :class:`MultiIndex` would be created instead of an :class:`Index` if a :class:`:meth:`core.window.rolling.RollingGroupby` object was created (:issue:`39732`) -- Bug in :meth:`DataFrameGroupBy.sample` where error was raised when ``weights`` was specified and the index was an :class:`Int64Index` (:issue:`39927`) -- Bug in :meth:`DataFrameGroupBy.aggregate` and :meth:`.Resampler.aggregate` would sometimes raise ``SpecificationError`` when passed a dictionary and columns were missing; will now always raise a ``KeyError`` instead (:issue:`40004`) -- Bug in :meth:`DataFrameGroupBy.sample` where column selection was not applied to sample result (:issue:`39928`) -- Bug in :class:`core.window.ewm.ExponentialMovingWindow` when calling ``__getitem__`` would incorrectly raise a ``ValueError`` when providing ``times`` (:issue:`40164`) -- Bug in :class:`core.window.ewm.ExponentialMovingWindow` when calling ``__getitem__`` would not retain ``com``, ``span``, ``alpha`` or ``halflife`` attributes (:issue:`40164`) -- :class:`core.window.ewm.ExponentialMovingWindow` now raises a ``NotImplementedError`` when specifying ``times`` with ``adjust=False`` due to an incorrect calculation (:issue:`40098`) -- Bug in :meth:`core.window.ewm.ExponentialMovingWindowGroupby.mean` where the times argument was ignored when ``engine='numba'`` (:issue:`40951`) -- Bug in :meth:`core.window.ewm.ExponentialMovingWindowGroupby.mean` where the wrong times were used in case of multiple groups (:issue:`40951`) -- Bug in :class:`core.window.ewm.ExponentialMovingWindowGroupby` where the times vector and values became out of sync for non-trivial groups (:issue:`40951`) -- Bug in :meth:`Series.asfreq` and :meth:`DataFrame.asfreq` dropping rows when the index is not sorted (:issue:`39805`) +- Bug in :meth:`Series.groupby` and :meth:`DataFrame.groupby` on an empty ``Series`` or ``DataFrame`` would lose index, columns, and/or data types when directly using the methods ``idxmax``, ``idxmin``, ``mad``, ``min``, ``max``, ``sum``, ``prod``, and ``skew`` or using them through ``apply``, ``aggregate``, or ``resample`` (:issue:`26411`) +- Bug in :meth:`.GroupBy.apply` where a :class:`MultiIndex` would be created instead of an :class:`Index` when used on a :class:`.RollingGroupby` object (:issue:`39732`) +- Bug in :meth:`.DataFrameGroupBy.sample` where an error was raised when ``weights`` was specified and the index was an :class:`Int64Index` (:issue:`39927`) +- Bug in :meth:`.DataFrameGroupBy.aggregate` and :meth:`.Resampler.aggregate` would sometimes raise a ``SpecificationError`` when passed a dictionary and columns were missing; will now always raise a ``KeyError`` instead (:issue:`40004`) +- Bug in :meth:`.DataFrameGroupBy.sample` where column selection was not applied to before computing the result (:issue:`39928`) +- Bug in :class:`.ExponentialMovingWindow` when calling ``__getitem__`` would incorrectly raise a ``ValueError`` when providing ``times`` (:issue:`40164`) +- Bug in :class:`.ExponentialMovingWindow` when calling ``__getitem__`` would not retain ``com``, ``span``, ``alpha`` or ``halflife`` attributes (:issue:`40164`) +- :class:`.ExponentialMovingWindow` now raises a ``NotImplementedError`` when specifying ``times`` with ``adjust=False`` due to an incorrect calculation (:issue:`40098`) +- Bug in :meth:`.ExponentialMovingWindowGroupby.mean` where the ``times`` argument was ignored when ``engine='numba'`` (:issue:`40951`) +- Bug in :meth:`.ExponentialMovingWindowGroupby.mean` where the wrong times were used the in case of multiple groups (:issue:`40951`) +- Bug in :class:`.ExponentialMovingWindowGroupby` where the times vector and values became out of sync for non-trivial groups (:issue:`40951`) +- Bug in :meth:`Series.asfreq` and :meth:`DataFrame.asfreq` dropping rows when the index was not sorted (:issue:`39805`) - Bug in aggregation functions for :class:`DataFrame` not respecting ``numeric_only`` argument when ``level`` keyword was given (:issue:`40660`) -- Bug in :meth:`SeriesGroupBy.aggregate` where using a user-defined function to aggregate a ``Series`` with an object-typed :class:`Index` causes an incorrect :class:`Index` shape (issue:`40014`) -- Bug in :class:`core.window.RollingGroupby` where ``as_index=False`` argument in ``groupby`` was ignored (:issue:`39433`) -- Bug in :meth:`.GroupBy.any` and :meth:`.GroupBy.all` raising ``ValueError`` when using with nullable type columns holding ``NA`` even with ``skipna=True`` (:issue:`40585`) -- Bug in :meth:`GroupBy.cummin` and :meth:`GroupBy.cummax` incorrectly rounding integer values near the ``int64`` implementations bounds (:issue:`40767`) -- Bug in :meth:`.GroupBy.rank` with nullable dtypes incorrectly raising ``TypeError`` (:issue:`41010`) +- Bug in :meth:`.SeriesGroupBy.aggregate` where using a user-defined function to aggregate a Series with an object-typed :class:`Index` causes an incorrect :class:`Index` shape (:issue:`40014`) +- Bug in :class:`.RollingGroupby` where ``as_index=False`` argument in ``groupby`` was ignored (:issue:`39433`) +- Bug in :meth:`.GroupBy.any` and :meth:`.GroupBy.all` raising a ``ValueError`` when using with nullable type columns holding ``NA`` even with ``skipna=True`` (:issue:`40585`) +- Bug in :meth:`.GroupBy.cummin` and :meth:`.GroupBy.cummax` incorrectly rounding integer values near the ``int64`` implementations bounds (:issue:`40767`) +- Bug in :meth:`.GroupBy.rank` with nullable dtypes incorrectly raising a ``TypeError`` (:issue:`41010`) - Bug in :meth:`.GroupBy.cummin` and :meth:`.GroupBy.cummax` computing wrong result with nullable data types too large to roundtrip when casting to float (:issue:`37493`) - Bug in :meth:`DataFrame.rolling` returning mean zero for all ``NaN`` window with ``min_periods=0`` if calculation is not numerical stable (:issue:`41053`) - Bug in :meth:`DataFrame.rolling` returning sum not zero for all ``NaN`` window with ``min_periods=0`` if calculation is not numerical stable (:issue:`41053`) -- Bug in :meth:`SeriesGroupBy.agg` failing to retain ordered :class:`CategoricalDtype` on order-preserving aggregations (:issue:`41147`) -- Bug in :meth:`DataFrameGroupBy.min` and :meth:`DataFrameGroupBy.max` with multiple object-dtype columns and ``numeric_only=False`` incorrectly raising ``ValueError`` (:issue:41111`) -- Bug in :meth:`DataFrameGroupBy.rank` with the GroupBy object's ``axis=0`` and the ``rank`` method's keyword ``axis=1`` (:issue:`41320`) +- Bug in :meth:`.SeriesGroupBy.agg` failing to retain ordered :class:`CategoricalDtype` on order-preserving aggregations (:issue:`41147`) +- Bug in :meth:`.GroupBy.min` and :meth:`.GroupBy.max` with multiple object-dtype columns and ``numeric_only=False`` incorrectly raising a ``ValueError`` (:issue:41111`) +- Bug in :meth:`.DataFrameGroupBy.rank` with the GroupBy object's ``axis=0`` and the ``rank`` method's keyword ``axis=1`` (:issue:`41320`) - Bug in :meth:`DataFrameGroupBy.__getitem__` with non-unique columns incorrectly returning a malformed :class:`SeriesGroupBy` instead of :class:`DataFrameGroupBy` (:issue:`41427`) -- Bug in :meth:`DataFrameGroupBy.transform` with non-unique columns incorrectly raising ``AttributeError`` (:issue:`41427`) -- Bug in :meth:`Resampler.apply` with non-unique columns incorrectly dropping duplicated columns (:issue:`41445`) -- Bug in :meth:`SeriesGroupBy` aggregations incorrectly returning empty :class:`Series` instead of raising ``TypeError`` on aggregations that are invalid for its dtype, e.g. ``.prod`` with ``datetime64[ns]`` dtype (:issue:`41342`) +- Bug in :meth:`.DataFrameGroupBy.transform` with non-unique columns incorrectly raising an ``AttributeError`` (:issue:`41427`) +- Bug in :meth:`.Resampler.apply` with non-unique columns incorrectly dropping duplicated columns (:issue:`41445`) +- Bug in :meth:`Series.groupby` aggregations incorrectly returning empty :class:`Series` instead of raising ``TypeError`` on aggregations that are invalid for its dtype, e.g. ``.prod`` with ``datetime64[ns]`` dtype (:issue:`41342`) - Bug in :meth:`DataFrame.rolling.__iter__` where ``on`` was not assigned to the index of the resulting objects (:issue:`40373`) -- Bug in :meth:`DataFrameGroupBy.transform` and :meth:`DataFrameGroupBy.agg` with ``engine="numba"`` where ``*args`` were being cached with the user passed function (:issue:`41647`) +- Bug in :meth:`.DataFrameGroupBy.transform` and :meth:`.DataFrameGroupBy.agg` with ``engine="numba"`` where ``*args`` were being cached with the user passed function (:issue:`41647`) Reshaping ^^^^^^^^^ -- Bug in :func:`merge` raising error when performing an inner join with partial index and ``right_index`` when no overlap between indices (:issue:`33814`) +- Bug in :func:`merge` raising error when performing an inner join with partial index and ``right_index=True`` when there was no overlap between indices (:issue:`33814`) - Bug in :meth:`DataFrame.unstack` with missing levels led to incorrect index names (:issue:`37510`) - Bug in :func:`merge_asof` propagating the right Index with ``left_index=True`` and ``right_on`` specification instead of left Index (:issue:`33463`) -- Bug in :func:`join` over :class:`MultiIndex` returned wrong result, when one of both indexes had only one level (:issue:`36909`) -- :meth:`merge_asof` raises ``ValueError`` instead of cryptic ``TypeError`` in case of non-numerical merge columns (:issue:`29130`) -- Bug in :meth:`DataFrame.join` not assigning values correctly when having :class:`MultiIndex` where at least one dimension is from dtype ``Categorical`` with non-alphabetically sorted categories (:issue:`38502`) -- :meth:`Series.value_counts` and :meth:`Series.mode` return consistent keys in original order (:issue:`12679`, :issue:`11227` and :issue:`39007`) -- Bug in :meth:`DataFrame.stack` not handling ``NaN`` in :class:`MultiIndex` columns correct (:issue:`39481`) -- Bug in :meth:`DataFrame.apply` would give incorrect results when used with a string argument and ``axis=1`` when the axis argument was not supported and now raises a ``ValueError`` instead (:issue:`39211`) -- Bug in :meth:`DataFrame.sort_values` not reshaping index correctly after sorting on columns, when ``ignore_index=True`` (:issue:`39464`) +- Bug in :meth:`DataFrame.join` on a DataFrame with a :class:`MultiIndex` returned the wrong result when one of both indexes had only one level (:issue:`36909`) +- :meth:`merge_asof` now raises a ``ValueError`` instead of a cryptic ``TypeError`` in case of non-numerical merge columns (:issue:`29130`) +- Bug in :meth:`DataFrame.join` not assigning values correctly when the DataFrame had a :class:`MultiIndex` and at least one dimension had dtype ``Categorical`` with non-alphabetically sorted categories (:issue:`38502`) +- :meth:`Series.value_counts` and :meth:`Series.mode` now return consistent keys in original order (:issue:`12679`, :issue:`11227` and :issue:`39007`) +- Bug in :meth:`DataFrame.stack` did not handle ``NaN`` in :class:`MultiIndex` columns correctly (:issue:`39481`) +- Bug in :meth:`DataFrame.apply` would give incorrect results when used with a string argument, ``axis=1``, and the axis argument was not supported; now raises a ``ValueError`` instead (:issue:`39211`) +- Bug in :meth:`DataFrame.sort_values` not reshaping the index correctly after sorting on columns when ``ignore_index=True`` (:issue:`39464`) - Bug in :meth:`DataFrame.append` returning incorrect dtypes with combinations of ``ExtensionDtype`` dtypes (:issue:`39454`) -- Bug in :meth:`DataFrame.append` returning incorrect dtypes with combinations of ``datetime64`` and ``timedelta64`` dtypes (:issue:`39574`) -- Bug in :meth:`DataFrame.pivot_table` returning a ``MultiIndex`` for a single value when operating on and empty ``DataFrame`` (:issue:`13483`) -- Allow :class:`Index` to be passed to the :func:`numpy.all` function (:issue:`40180`) -- Bug in :meth:`DataFrame.stack` not preserving ``CategoricalDtype`` in a ``MultiIndex`` (:issue:`36991`) -- Bug in :func:`to_datetime` raising error when input sequence contains unhashable items (:issue:`39756`) -- Bug in :meth:`Series.explode` preserving index when ``ignore_index`` was ``True`` and values were scalars (:issue:`40487`) -- Bug in :func:`to_datetime` raising ``ValueError`` when :class:`Series` contains ``None`` and ``NaT`` and has more than 50 elements (:issue:`39882`) +- Bug in :meth:`DataFrame.append` returning incorrect dtypes when used with combinations of ``datetime64`` and ``timedelta64`` dtypes (:issue:`39574`) +- Bug in :meth:`DataFrame.pivot_table` returning a :class:`MultiIndex` for a single value when operating on an empty DataFrame (:issue:`13483`) +- :class:`Index` can now be passed to the :func:`numpy.all` function (:issue:`40180`) +- Bug in :meth:`DataFrame.stack` not preserving ``CategoricalDtype`` in a :class:`MultiIndex` (:issue:`36991`) +- Bug in :func:`to_datetime` raising an error when the input sequence contained unhashable items (:issue:`39756`) +- Bug in :meth:`Series.explode` preserving the index when ``ignore_index`` was ``True`` and values were scalars (:issue:`40487`) +- Bug in :func:`to_datetime` raising a ``ValueError`` when :class:`Series` contains ``None`` and ``NaT`` and has more than 50 elements (:issue:`39882`) Sparse ^^^^^^ -- Bug in :meth:`DataFrame.sparse.to_coo` raising ``KeyError`` with columns that are a numeric :class:`Index` without a 0 (:issue:`18414`) +- Bug in :meth:`DataFrame.sparse.to_coo` raising a ``KeyError`` with columns that are a numeric :class:`Index` without a ``0`` (:issue:`18414`) - Bug in :meth:`SparseArray.astype` with ``copy=False`` producing incorrect results when going from integer dtype to floating dtype (:issue:`34456`) -- Implemented :meth:`SparseArray.max` and :meth:`SparseArray.min` (:issue:`40921`) +- Bug in :meth:`SparseArray.max` and :meth:`SparseArray.min` would always return an empty result (:issue:`40921`) ExtensionArray ^^^^^^^^^^^^^^ -- Bug in :meth:`DataFrame.where` when ``other`` is a :class:`Series` with :class:`ExtensionArray` dtype (:issue:`38729`) -- Fixed bug where :meth:`Series.idxmax`, :meth:`Series.idxmin` and ``argmax/min`` fail when the underlying data is :class:`ExtensionArray` (:issue:`32749`, :issue:`33719`, :issue:`36566`) -- Fixed a bug where some properties of subclasses of :class:`PandasExtensionDtype` where improperly cached (:issue:`40329`) -- Bug in :meth:`DataFrame.mask` where masking a :class:`Dataframe` with an :class:`ExtensionArray` dtype raises ``ValueError`` (:issue:`40941`) +- Bug in :meth:`DataFrame.where` when ``other`` is a Series with :class:`ExtensionArray` dtype (:issue:`38729`) +- Fixed bug where :meth:`Series.idxmax`, :meth:`Series.idxmin`, :meth:`Series.argmax`, and :meth:`Series.argmin` would fail when the underlying data is an :class:`ExtensionArray` (:issue:`32749`, :issue:`33719`, :issue:`36566`) +- Fixed bug where some properties of subclasses of :class:`PandasExtensionDtype` where improperly cached (:issue:`40329`) +- Bug in :meth:`DataFrame.mask` where masking a DataFrame with an :class:`ExtensionArray` dtype raises ``ValueError`` (:issue:`40941`) Styler ^^^^^^ -- Bug in :class:`Styler` where ``subset`` arg in methods raised an error for some valid multiindex slices (:issue:`33562`) -- :class:`Styler` rendered HTML output minor alterations to support w3 good code standard (:issue:`39626`) +- Bug in :class:`Styler` where the ``subset`` argument in methods raised an error for some valid MultiIndex slices (:issue:`33562`) +- :class:`Styler` rendered HTML output has seen minor alterations to support w3 good code standards (:issue:`39626`) - Bug in :class:`Styler` where rendered HTML was missing a column class identifier for certain header cells (:issue:`39716`) -- Bug in :meth:`Styler.background_gradient` where text-color was not determined correctly (:issue:`39888`) +- Bug in :meth:`.Styler.background_gradient` where text-color was not determined correctly (:issue:`39888`) - Bug in :class:`Styler` where multiple elements in CSS-selectors were not correctly added to ``table_styles`` (:issue:`39942`) -- Bug in :class:`.Styler` where copying from Jupyter dropped top left cell and misaligned headers (:issue:`12147`) +- Bug in :class:`.Styler` where copying from Jupyter dropped the top left cell and misaligned headers (:issue:`12147`) - Bug in :class:`.Styler.where` where ``kwargs`` were not passed to the applicable callable (:issue:`40845`) -- Bug in :class:`Styler` which caused CSS to duplicate on multiple renders. (:issue:`39395`, :issue:`40334`) +- Bug in :class:`Styler` caused CSS to duplicate on multiple renders (:issue:`39395`, :issue:`40334`) Other ^^^^^ - Bug in :class:`Index` constructor sometimes silently ignoring a specified ``dtype`` (:issue:`38879`) -- Bug in :func:`pandas.api.types.infer_dtype` not recognizing Series, Index or array with a period dtype (:issue:`23553`) -- Bug in :func:`pandas.api.types.infer_dtype` raising an error for general :class:`.ExtensionArray` objects. It will now return ``"unknown-array"`` instead of raising (:issue:`37367`) +- Bug in :func:`.infer_dtype` not recognizing Series, Index, or array with a Period dtype (:issue:`23553`) +- Bug in :func:`.infer_dtype` raising an error for general :class:`.ExtensionArray` objects. It will now return ``"unknown-array"`` instead of raising (:issue:`37367`) - Bug in constructing a :class:`Series` from a list and a :class:`PandasDtype` (:issue:`39357`) - ``inspect.getmembers(Series)`` no longer raises an ``AbstractMethodError`` (:issue:`38782`) -- Bug in :meth:`Series.where` with numeric dtype and ``other = None`` not casting to ``nan`` (:issue:`39761`) +- Bug in :meth:`Series.where` with numeric dtype and ``other=None`` not casting to ``nan`` (:issue:`39761`) - :meth:`Index.where` behavior now mirrors :meth:`Index.putmask` behavior, i.e. ``index.where(mask, other)`` matches ``index.putmask(~mask, other)`` (:issue:`39412`) - Bug in :func:`pandas.testing.assert_series_equal`, :func:`pandas.testing.assert_frame_equal`, :func:`pandas.testing.assert_index_equal` and :func:`pandas.testing.assert_extension_array_equal` incorrectly raising when an attribute has an unrecognized NA type (:issue:`39461`) - Bug in :func:`pandas.testing.assert_index_equal` with ``exact=True`` not raising when comparing :class:`CategoricalIndex` instances with ``Int64Index`` and ``RangeIndex`` categories (:issue:`41263`) -- Bug in :meth:`DataFrame.equals`, :meth:`Series.equals`, :meth:`Index.equals` with object-dtype containing ``np.datetime64("NaT")`` or ``np.timedelta64("NaT")`` (:issue:`39650`) -- Bug in :func:`pandas.util.show_versions` where console JSON output was not proper JSON (:issue:`39701`) -- Let Pandas compile on z/OS when using `xlc `_ (:issue:`35826`) -- Bug in :meth:`DataFrame.convert_dtypes` incorrectly raised ValueError when called on an empty DataFrame (:issue:`40393`) +- Bug in :meth:`DataFrame.equals`, :meth:`Series.equals`, and :meth:`Index.equals` with object-dtype containing ``np.datetime64("NaT")`` or ``np.timedelta64("NaT")`` (:issue:`39650`) +- Bug in :func:`show_versions` where console JSON output was not proper JSON (:issue:`39701`) +- pandas can now compile on z/OS when using `xlc `_ (:issue:`35826`) +- Bug in :meth:`DataFrame.convert_dtypes` incorrectly raised a ``ValueError`` when called on an empty DataFrame (:issue:`40393`) - Bug in :meth:`DataFrame.agg()` not sorting the aggregated axis in the order of the provided aggragation functions when one or more aggregation function fails to produce results (:issue:`33634`) - Bug in :meth:`DataFrame.clip` not interpreting missing values as no threshold (:issue:`40420`) - Bug in :class:`Series` backed by :class:`DatetimeArray` or :class:`TimedeltaArray` sometimes failing to set the array's ``freq`` to ``None`` (:issue:`41425`) From 41f87d93c4786cdbf6697fc8d13de191511144ae Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Mon, 31 May 2021 14:21:15 -0400 Subject: [PATCH 2/9] Few more fixups --- doc/source/whatsnew/v1.3.0.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index 12ecd341fdff5..a65f98a115c0c 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -118,14 +118,14 @@ Styler Improvements We provided some focused development on :class:`.Styler`, including altering methods to accept more universal CSS language for arguments, such as ``'color:red;'`` instead of ``[('color', 'red')]`` (:issue:`39564`). This is also added to the built-in methods -to allow custom CSS highlighting instead of default background coloring (:issue:`40242`). -Enhancements to other built-in methods include extending the :meth:`.Styler.background_gradient` -method to shade elements based on a given gradient map and not be restricted only to +now allowing custom CSS highlighting instead of the default background coloring (:issue:`40242`). +Enhancements to other built-in methods include extending :meth:`.Styler.background_gradient` +to shade elements based on a given gradient map and not be restricted only to values in the DataFrame (:issue:`39930`, :issue:`22727`, :issue:`28901`). Additional built-in methods such as :meth:`.Styler.highlight_between`, :meth:`.Styler.highlight_quantile` and :meth:`.Styler.text_gradient` have been added (:issue:`39821`, :issue:`40926`, :issue:`41098`). -The :meth:`.Styler.apply` method now consistently accepts functions with ``ndarray`` output to +The :meth:`.Styler.apply` method now consistently accepts functions that return an ``ndarray`` to allow more flexible development of UDFs when ``axis`` is ``None``, ``0``, or ``1`` (:issue:`39393`). :meth:`.Styler.set_tooltips` is a new method that allows adding hover tooltips to @@ -141,12 +141,12 @@ precision, and perform HTML escaping (:issue:`40437`, :issue:`40134`). There hav properly format HTML and eliminate some inconsistencies (:issue:`39942`, :issue:`40356`, :issue:`39807`, :issue:`39889`, :issue:`39627`). Many features of the :class:`.Styler` class are now either partially or fully usable on a DataFrame with a non-unique indexes or columns (:issue:`41269`). -One also has greater control of the display through separate sparsification of the index or columns using the :ref:`new 'styler' options context ` (:issue:`41142`). +One also has greater control of the display through separate sparsification of the index or columns using the :ref:`new styler options context ` (:issue:`41142`). We have added an extension to allow LaTeX styling as an alternative to CSS styling and the method :meth:`.Styler.to_latex` that renders the necessary LaTeX format including built-up styles. An additional file IO function :meth:`.Styler.to_html` has been added for convenience (:issue:`40312`). -The :ref:`Styler documentation <../user_guide/style.ipynb>`_ has also seen major revisions in light of new features (:issue:`39720`, :issue:`39317`, :issue:`40493`). +The `Styler documentation <../user_guide/style.ipynb>`_ has also seen major revisions in light of the new features (:issue:`39720`, :issue:`39317`, :issue:`40493`). .. _whatsnew_130.dataframe_honors_copy_with_dict: From 468aaac72b9cb0037f0339346bcea2994c4d0cd4 Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Mon, 31 May 2021 23:27:29 -0400 Subject: [PATCH 3/9] Reworked the Styler section --- doc/source/whatsnew/v1.3.0.rst | 61 +++++++++++++++------------------- 1 file changed, 27 insertions(+), 34 deletions(-) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index a65f98a115c0c..c2d1536ecb2bc 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -110,43 +110,36 @@ both XPath 1.0 and XSLT 1.0 are available. (:issue:`27554`) For more, see :ref:`io.xml` in the user guide on IO tools. -.. _whatsnew_130.styler_upgrades: +.. _whatsnew_130.styler_enhancements: -Styler Improvements +Styler enhancements ^^^^^^^^^^^^^^^^^^^ -We provided some focused development on :class:`.Styler`, including altering methods -to accept more universal CSS language for arguments, such as ``'color:red;'`` instead of -``[('color', 'red')]`` (:issue:`39564`). This is also added to the built-in methods -now allowing custom CSS highlighting instead of the default background coloring (:issue:`40242`). -Enhancements to other built-in methods include extending :meth:`.Styler.background_gradient` -to shade elements based on a given gradient map and not be restricted only to -values in the DataFrame (:issue:`39930`, :issue:`22727`, :issue:`28901`). Additional -built-in methods such as :meth:`.Styler.highlight_between`, :meth:`.Styler.highlight_quantile` -and :meth:`.Styler.text_gradient` have been added (:issue:`39821`, :issue:`40926`, :issue:`41098`). - -The :meth:`.Styler.apply` method now consistently accepts functions that return an ``ndarray`` to -allow more flexible development of UDFs when ``axis`` is ``None``, ``0``, or ``1`` (:issue:`39393`). - -:meth:`.Styler.set_tooltips` is a new method that allows adding hover tooltips to -enhance interactive displays (:issue:`35643`). :meth:`.Styler.set_td_classes`, which was recently -introduced in v1.2.0 (:issue:`36159`) to allow adding specific CSS classes to data cells, has -been made as performant as :meth:`.Styler.apply` and :meth:`.Styler.applymap` (:issue:`40453`), -if not more performant in some cases. The overall performance of HTML -render times has been considerably improved to -match :meth:`DataFrame.to_html` (:issue:`39952`, :issue:`37792`, :issue:`40425`). - -The :meth:`.Styler.format` has been improved to easily format missing data, -precision, and perform HTML escaping (:issue:`40437`, :issue:`40134`). There have been numerous other bug fixes to -properly format HTML and eliminate some inconsistencies (:issue:`39942`, :issue:`40356`, :issue:`39807`, :issue:`39889`, :issue:`39627`). - -Many features of the :class:`.Styler` class are now either partially or fully usable on a DataFrame with a non-unique indexes or columns (:issue:`41269`). -One also has greater control of the display through separate sparsification of the index or columns using the :ref:`new styler options context ` (:issue:`41142`). - -We have added an extension to allow LaTeX styling as an alternative to CSS styling and the method :meth:`.Styler.to_latex` -that renders the necessary LaTeX format including built-up styles. An additional file IO function :meth:`.Styler.to_html` has been added for convenience (:issue:`40312`). - -The `Styler documentation <../user_guide/style.ipynb>`_ has also seen major revisions in light of the new features (:issue:`39720`, :issue:`39317`, :issue:`40493`). +We provided some focused development on :class:`.Styler`. See also the `Styler documentation <../user_guide/style.ipynb>`_ +which has also been revised and improved(:issue:`39720`, :issue:`39317`, :issue:`40493`). + + - The method :meth:`.Styler.set_table_styles` can now accept more natural CSS language for arguments, such as ``'color:red;'`` instead of ``[('color', 'red')]`` (:issue:`39563`) + - The methods :meth:`.Styler.highlight_null`, :meth:`.Styler.highlight_min`, and :meth:`.Styler.highlight_max` now allow custom CSS highlighting instead of the default background coloring (:issue:`40242`) + - Extended :meth:`.Styler.background_gradient` to shade elements based on a given gradient map and not be restricted to values in the DataFrame (:issue:`22727`, :issue:`28901`) + - Added the method :meth:`.Styler.highlight_between` (:issue:`39821`) + - Added the method :meth:`.Styler.highlight_quantile` (:issue:`40926`) + - Added the method :meth:`.Styler.text_gradient` (:issue:`41098`) + - The :meth:`.Styler.apply` method now consistently accepts functions that return an ``ndarray`` to allow more flexible development of UDFs when ``axis`` is ``None``, ``0``, or ``1`` (:issue:`39359`) + - Added the method :meth:`.Styler.set_tooltips` allowing hover tooltips; this can be used enhance interactive displays (:issue:`21266`) + - The method :meth:`.Styler.set_td_classes` is now as performant as :meth:`.Styler.apply` and :meth:`.Styler.applymap`, and more so in some cases (:issue:`40453`) + - Considerably improved performance of HTML render times and now matches :meth:`DataFrame.to_html` (:issue:`39952`, :issue:`40425`) + - Added the parameter ``escape`` to the method :meth:`.Styler.format` to escape data before it is passed to the formatter (:issue:`40388`) + - Added the parameter ``precision`` to the method :meth:`.Styler.format` to control the display of floating point numbers (:issue:`40134`) + - Fixed bug in the method :meth:`.Styler.set_table_styles` where multiple elements in CSS-selectors of the ``table_styles`` argument were not correctly added (:issue:`34061`) + - Fixed bug in :class:`.Styler` where copying the output from Jupyter dropped the top left cell and misaligned headers (:issue:`12147`) + - Fixed bug in :class:`.Styler` where rendered HTML was missing a column class identifier for certain header cells (:issue:`39716`) + - Fixed bug in :meth:`.Styler.background_gradient` where the text-color was not determined correctly (:issue:`39888`) + - :class:`.Styler` rendered HTML output now follows the `w3 HTML Style Guide `_ (:issue:`39626`) + - Many features of the :class:`.Styler` class are now either partially or fully usable on a DataFrame with a non-unique indexes or columns (:issue:`41143`) + - One has greater control of the display through separate sparsification of the index or columns using the :ref:`new styler options `, which are also usable via :func:`option_context` (:issue:`41142`) + - Added the method :meth:`.Styler.to_latex` (:issue:`21673`) + - Added the method :meth:`.Styler.to_html` (:issue:`13379`) + .. _whatsnew_130.dataframe_honors_copy_with_dict: From d3a7bb3fa331a4df99900b19dea56da2023cc1d7 Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Sat, 5 Jun 2021 16:40:49 -0400 Subject: [PATCH 4/9] Re-organize and dedupe Styler notes --- doc/source/whatsnew/v1.3.0.rst | 40 ++++++++++------------------------ 1 file changed, 11 insertions(+), 29 deletions(-) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index df78d28c620ef..2053f0d36d58a 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -116,39 +116,29 @@ Styler enhancements ^^^^^^^^^^^^^^^^^^^ We provided some focused development on :class:`.Styler`. See also the `Styler documentation <../user_guide/style.ipynb>`_ -which has also been revised and improved(:issue:`39720`, :issue:`39317`, :issue:`40493`). +which has been revised and improved (:issue:`39720`, :issue:`39317`, :issue:`40493`). - The method :meth:`.Styler.set_table_styles` can now accept more natural CSS language for arguments, such as ``'color:red;'`` instead of ``[('color', 'red')]`` (:issue:`39563`) - The methods :meth:`.Styler.highlight_null`, :meth:`.Styler.highlight_min`, and :meth:`.Styler.highlight_max` now allow custom CSS highlighting instead of the default background coloring (:issue:`40242`) - - Extended :meth:`.Styler.background_gradient` to shade elements based on a given gradient map and not be restricted to values in the DataFrame (:issue:`22727`, :issue:`28901`) + - :meth:`.Styler.set_tooltips` allows on hover tooltips to be added to styled HTML dataframes (:issue:`35643`, :issue:`21266`, :issue:`39317`, :issue:`39708`, :issue:`40284`) +- :meth:`.Styler.apply` now accepts functions that return an ``ndarray`` when ``axis=None``, making it now consistent with the ``axis=0`` and ``axis=1`` behavior (:issue:`39359`) +- When incorrectly formatted CSS is given via :meth:`.Styler.apply` or :meth:`.Styler.applymap`, an error is now raised upon rendering (:issue:`39660`) + - :meth:`.Styler.format` now accepts the keyword argument ``escape`` for optional HTML escaping (:issue:`40437`) +- :meth:`.Styler.background_gradient` has gained the argument ``gmap`` to supply a specific gradient map for shading (:issue:`22727`) +- :meth:`.Styler.clear` now clears :attr:`Styler.hidden_index` and :attr:`Styler.hidden_columns` as well (:issue:`40484`) - Added the method :meth:`.Styler.highlight_between` (:issue:`39821`) - Added the method :meth:`.Styler.highlight_quantile` (:issue:`40926`) - Added the method :meth:`.Styler.text_gradient` (:issue:`41098`) - - The :meth:`.Styler.apply` method now consistently accepts functions that return an ``ndarray`` to allow more flexible development of UDFs when ``axis`` is ``None``, ``0``, or ``1`` (:issue:`39359`) - Added the method :meth:`.Styler.set_tooltips` allowing hover tooltips; this can be used enhance interactive displays (:issue:`21266`) - - The method :meth:`.Styler.set_td_classes` is now as performant as :meth:`.Styler.apply` and :meth:`.Styler.applymap`, and more so in some cases (:issue:`40453`) - - Considerably improved performance of HTML render times and now matches :meth:`DataFrame.to_html` (:issue:`39952`, :issue:`40425`) - Added the parameter ``escape`` to the method :meth:`.Styler.format` to escape data before it is passed to the formatter (:issue:`40388`) - Added the parameter ``precision`` to the method :meth:`.Styler.format` to control the display of floating point numbers (:issue:`40134`) - - Fixed bug in the method :meth:`.Styler.set_table_styles` where multiple elements in CSS-selectors of the ``table_styles`` argument were not correctly added (:issue:`34061`) - - Fixed bug in :class:`.Styler` where copying the output from Jupyter dropped the top left cell and misaligned headers (:issue:`12147`) - - Fixed bug in :class:`.Styler` where rendered HTML was missing a column class identifier for certain header cells (:issue:`39716`) - - Fixed bug in :meth:`.Styler.background_gradient` where the text-color was not determined correctly (:issue:`39888`) - :class:`.Styler` rendered HTML output now follows the `w3 HTML Style Guide `_ (:issue:`39626`) - Many features of the :class:`.Styler` class are now either partially or fully usable on a DataFrame with a non-unique indexes or columns (:issue:`41143`) - One has greater control of the display through separate sparsification of the index or columns using the :ref:`new styler options `, which are also usable via :func:`option_context` (:issue:`41142`) +- Added the option ``sytler.max.elements`` to avoid browser overload when styling large DataFrames (:issue:`40712`) - Added the method :meth:`.Styler.to_latex` (:issue:`21673`) - Added the method :meth:`.Styler.to_html` (:issue:`13379`) -:class:`.Styler` has also been compatible with non-unique index or columns, at least for as many features as are fully compatible, others made only partially compatible (:issue:`41269`). -One also has greater control of the display through separate sparsification of the index or columns, using the new 'styler' options context (:issue:`41142`). -Render trimming has also been added for large numbers of data elements to avoid browser overload (:issue:`40712`). - -We have added an extension to allow LaTeX styling as an alternative to CSS styling and a method :meth:`.Styler.to_latex` -which renders the necessary LaTeX format including built-up styles (:issue:`21673`, :issue:`41659`). An additional file io function :meth:`Styler.to_html` has been added for convenience (:issue:`40312`). - -Documentation has also seen major revisions in light of new features (:issue:`39720` :issue:`39317` :issue:`40493`) - .. _whatsnew_130.dataframe_honors_copy_with_dict: DataFrame constructor honors ``copy=False`` with dict @@ -213,15 +203,6 @@ Other enhancements - Passing a :class:`DataFrame` indexer to ``iloc`` is now disallowed for :meth:`Series.__getitem__` and :meth:`DataFrame.__getitem__` (:issue:`39004`) - :meth:`Series.apply` can now accept list-like or dictionary-like arguments that aren't lists or dictionaries, e.g. ``ser.apply(np.array(["sum", "mean"]))``, which was already the case for :meth:`DataFrame.apply` (:issue:`39140`) - :meth:`DataFrame.plot.scatter` can now accept a categorical column for the argument ``c`` (:issue:`12380`, :issue:`31357`) -- :meth:`.Styler.set_tooltips` allows on hover tooltips to be added to styled HTML dataframes (:issue:`35643`, :issue:`21266`, :issue:`39317`, :issue:`39708`, :issue:`40284`) -- :meth:`.Styler.set_table_styles` amended to optionally allow certain CSS input arguments as strings (:issue:`39564`) -- :meth:`.Styler.apply` now accepts functions that return an ``ndarray`` when ``axis=None``, consistent with the ``axis=0`` and ``axis=1`` behavior (:issue:`39359`) -- :meth:`.Styler.apply` and :meth:`.Styler.applymap` now raise errors if incorrectly formatted CSS is passed (:issue:`39660`) -- :meth:`.Styler.format` now accepts the keyword argument ``escape`` for optional HTML escaping (:issue:`40437`) -- :meth:`.Styler.background_gradient` has gained the argument ``gmap`` to supply a specific gradient map (:issue:`22727`) -- :meth:`.Styler.clear` now clears :attr:`Styler.hidden_index` and :attr:`Styler.hidden_columns` as well (:issue:`40484`) -- Builtin highlighting methods in :class:`.Styler` have a more consistent signature and css customisability (:issue:`40242`) -- Added the method :meth:`.Styler.highlight_between` (:issue:`39821`) - :meth:`Series.loc` now raises a helpful error message when the Series has a :class:`MultiIndex` and the indexer has too many dimensions (:issue:`35349`) - :func:`read_stata` now supports reading data from compressed files (:issue:`26599`) - Added support for parsing ``ISO 8601``-like timestamps with negative signs to :class:`Timedelta` (:issue:`37172`) @@ -808,7 +789,8 @@ Performance improvements - Performance improvement in :func:`unique` for object data type (:issue:`37615`) - Performance improvement in :func:`json_normalize` for basic cases (including separators) (:issue:`40035` :issue:`15621`) - Performance improvement in :class:`.ExpandingGroupby` aggregation methods (:issue:`39664`) -- Performance improvement in :class:`.Styler` where render times are more than 50% reduced (:issue:`39972` :issue:`39952`) +- Performance improvement in :class:`.Styler` where render times are more than 50% reduced and now matches :meth:`DataFrame.to_html` (:issue:`39972` :issue:`39952`, :issue:`40425`) +- The method :meth:`.Styler.set_td_classes` is now as performant as :meth:`.Styler.apply` and :meth:`.Styler.applymap`, and even more so in some cases (:issue:`40453`) - Performance improvement in :meth:`.ExponentialMovingWindow.mean` with ``times`` (:issue:`39784`) - Performance improvement in :meth:`.GroupBy.apply` when requiring the python fallback implementation (:issue:`40176`) - Performance improvement in the conversion of a PyArrow Boolean array to a pandas nullable Boolean array (:issue:`41051`) @@ -1115,7 +1097,7 @@ Styler - :class:`.Styler` rendered HTML output has seen minor alterations to support w3 good code standards (:issue:`39626`) - Bug in :class:`.Styler` where rendered HTML was missing a column class identifier for certain header cells (:issue:`39716`) - Bug in :meth:`.Styler.background_gradient` where text-color was not determined correctly (:issue:`39888`) -- Bug in :class:`.Styler` where multiple elements in CSS-selectors were not correctly added to ``table_styles`` (:issue:`39942`) +- Bug in :meth:`.Styler.set_table_styles` where multiple elements in CSS-selectors of the ``table_styles`` argument were not correctly added (:issue:`34061`) - Bug in :class:`.Styler` where copying from Jupyter dropped the top left cell and misaligned headers (:issue:`12147`) - Bug in :class:`Styler.where` where ``kwargs`` were not passed to the applicable callable (:issue:`40845`) - Bug in :class:`.Styler` caused CSS to duplicate on multiple renders (:issue:`39395`, :issue:`40334`) From 6adf94fa76434a381313fc3bef3721d58a96b6e1 Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Mon, 7 Jun 2021 16:41:17 -0400 Subject: [PATCH 5/9] Some reverts and fixups --- doc/source/whatsnew/v1.3.0.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index 6ad2091987ffb..980b0dc8677f1 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -193,7 +193,7 @@ Other enhancements - :func:`read_sql_query` now accepts a ``dtype`` argument to cast the columnar data from the SQL database based on user input (:issue:`10285`) - Improved integer type mapping from pandas to SQLAlchemy when using :meth:`DataFrame.to_sql` (:issue:`35076`) - :func:`to_numeric` now supports downcasting of nullable ``ExtensionDtype`` objects (:issue:`33013`) -- Add support for dict-like names in :meth:`Index.set_names` and :meth:`.Index.rename` in the case of a :class:`MultiIndex` (:issue:`20421`) +- Added support for dict-like names in :class:`MultiIndex.set_names` and :class:`MultiIndex.rename` (:issue:`20421`) - :func:`read_excel` can now auto-detect .xlsb files and older .xls files (:issue:`35416`, :issue:`41225`) - :class:`ExcelWriter` now accepts an ``if_sheet_exists`` parameter to control the behaviour of append mode when writing to existing sheets (:issue:`40230`) - :meth:`.Rolling.sum`, :meth:`.Expanding.sum`, :meth:`.Rolling.mean`, :meth:`.Expanding.mean`, :meth:`.ExponentialMovingWindow.mean`, :meth:`.Rolling.median`, :meth:`.Expanding.median`, :meth:`.Rolling.max`, :meth:`.Expanding.max`, :meth:`.Rolling.min`, and :meth:`.Expanding.min` now support `Numba `_ execution with the ``engine`` keyword (:issue:`38895`, :issue:`41267`) @@ -659,9 +659,10 @@ Deprecations - Deprecated support for ``np.ma.mrecords.MaskedRecords`` in the :class:`DataFrame` constructor, pass ``{name: data[name] for name in data.dtype.names}`` instead (:issue:`40363`) - Deprecated using :func:`merge`, :meth:`DataFrame.merge`, and :meth:`DataFrame.join` on a different number of levels (:issue:`34862`) - Deprecated the use of ``**kwargs`` in :class:`.ExcelWriter`; use the keyword argument ``engine_kwargs`` instead (:issue:`40430`) -- Deprecated the ``level`` keyword for :class:`DataFrame` and :class:`Series` aggregations (e.g. :meth:`DataFrame.sum`); use groupby instead (:issue:`39983`) +- Deprecated the ``level`` keyword for :class:`DataFrame` and :class:`Series` aggregations; use groupby instead (:issue:`39983`) - Deprecated the ``inplace`` parameter of :meth:`Categorical.remove_categories`, :meth:`Categorical.add_categories`, :meth:`Categorical.reorder_categories`, :meth:`Categorical.rename_categories`, :meth:`Categorical.set_categories` and will be removed in a future version (:issue:`37643`) - Deprecated :func:`merge` producing duplicated columns through the ``suffixes`` keyword and already existing columns (:issue:`22818`) +- Deprecated setting :attr:`Categorical._codes`, create a new :class:`Categorical` with the desired codes instead (:issue:`40606`) - Deprecated the ``convert_float`` optional argument in :func:`read_excel` and :meth:`ExcelFile.parse` (:issue:`41127`) - Deprecated behavior of :meth:`DatetimeIndex.union` with mixed timezones; in a future version both will be cast to UTC instead of object dtype (:issue:`39328`) - Deprecated using ``usecols`` with out of bounds indices for :func:`read_csv` with ``engine="c"`` (:issue:`25623`) @@ -907,7 +908,7 @@ Indexing - Bug in :meth:`DataFrame.reindex` and :meth:`Series.reindex` with timezone aware indexes raising a ``TypeError`` for ``method="ffill"`` and ``method="bfill"`` and specified ``tolerance`` (:issue:`38566`) - Bug in :meth:`DataFrame.reindex` with ``datetime64[ns]`` or ``timedelta64[ns]`` incorrectly casting to integers when the ``fill_value`` requires casting to object dtype (:issue:`39755`) - Bug in :meth:`DataFrame.__setitem__` raising a ``ValueError`` when setting on an empty :class:`DataFrame` using specified columns and a nonempty :class:`DataFrame` value (:issue:`38831`) -- Bug in :meth:`DataFrame.loc.__setitem__` raising ValueError when operating on a unique column when the :class:`DataFrame` has duplicate columns (:issue:`38521`) +- Bug in :meth:`DataFrame.loc.__setitem__` raising a ``ValueError`` when operating on a unique column when the :class:`DataFrame` has duplicate columns (:issue:`38521`) - Bug in :meth:`DataFrame.iloc.__setitem__` and :meth:`DataFrame.loc.__setitem__` with mixed dtypes when setting with a dictionary value (:issue:`38335`) - Bug in :meth:`Series.loc.__setitem__` and :meth:`DataFrame.loc.__setitem__` raising ``KeyError`` when provided a Boolean generator (:issue:`39614`) - Bug in :meth:`Series.iloc` and :meth:`DataFrame.iloc` raising a ``KeyError`` when provided a generator (:issue:`39614`) @@ -1089,10 +1090,10 @@ Sparse ExtensionArray ^^^^^^^^^^^^^^ -- Bug in :meth:`DataFrame.where` when ``other`` is a Series with :class:`ExtensionArray` dtype (:issue:`38729`) +- Bug in :meth:`DataFrame.where` when ``other`` is a Series with an :class:`ExtensionDtype` (:issue:`38729`) - Fixed bug where :meth:`Series.idxmax`, :meth:`Series.idxmin`, :meth:`Series.argmax`, and :meth:`Series.argmin` would fail when the underlying data is an :class:`ExtensionArray` (:issue:`32749`, :issue:`33719`, :issue:`36566`) - Fixed bug where some properties of subclasses of :class:`PandasExtensionDtype` where improperly cached (:issue:`40329`) -- Bug in :meth:`DataFrame.mask` where masking a DataFrame with an :class:`ExtensionArray` dtype raises ``ValueError`` (:issue:`40941`) +- Bug in :meth:`DataFrame.mask` where masking a DataFrame with an :class:`ExtensionDtype` raises a ``ValueError`` (:issue:`40941`) Styler ^^^^^^ From 0d01c9e03c672df2fc5f5588e52297ba27254aec Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Mon, 7 Jun 2021 16:45:22 -0400 Subject: [PATCH 6/9] caused -> causing --- doc/source/whatsnew/v1.3.0.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index 980b0dc8677f1..119d7fc090f5a 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -1105,7 +1105,7 @@ Styler - Bug in :meth:`.Styler.set_table_styles` where multiple elements in CSS-selectors of the ``table_styles`` argument were not correctly added (:issue:`34061`) - Bug in :class:`.Styler` where copying from Jupyter dropped the top left cell and misaligned headers (:issue:`12147`) - Bug in :class:`Styler.where` where ``kwargs`` were not passed to the applicable callable (:issue:`40845`) -- Bug in :class:`.Styler` caused CSS to duplicate on multiple renders (:issue:`39395`, :issue:`40334`) +- Bug in :class:`.Styler` causing CSS to duplicate on multiple renders (:issue:`39395`, :issue:`40334`) Other From d87b59d8c10b4cafe85d5a40a26d01cf159321b2 Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Tue, 8 Jun 2021 23:15:42 -0400 Subject: [PATCH 7/9] Fixups --- doc/source/whatsnew/v1.3.0.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index 119d7fc090f5a..d62cf11ddd44e 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -120,22 +120,20 @@ which has been revised and improved (:issue:`39720`, :issue:`39317`, :issue:`404 - The method :meth:`.Styler.set_table_styles` can now accept more natural CSS language for arguments, such as ``'color:red;'`` instead of ``[('color', 'red')]`` (:issue:`39563`) - The methods :meth:`.Styler.highlight_null`, :meth:`.Styler.highlight_min`, and :meth:`.Styler.highlight_max` now allow custom CSS highlighting instead of the default background coloring (:issue:`40242`) - - :meth:`.Styler.set_tooltips` allows on hover tooltips to be added to styled HTML dataframes (:issue:`35643`, :issue:`21266`, :issue:`39317`, :issue:`39708`, :issue:`40284`) - :meth:`.Styler.apply` now accepts functions that return an ``ndarray`` when ``axis=None``, making it now consistent with the ``axis=0`` and ``axis=1`` behavior (:issue:`39359`) - When incorrectly formatted CSS is given via :meth:`.Styler.apply` or :meth:`.Styler.applymap`, an error is now raised upon rendering (:issue:`39660`) - - :meth:`.Styler.format` now accepts the keyword argument ``escape`` for optional HTML escaping (:issue:`40437`) + - :meth:`.Styler.format` now accepts the keyword argument ``escape`` for optional HTML and LaTex escaping (:issue:`40388`, :issue:`41619`) - :meth:`.Styler.background_gradient` has gained the argument ``gmap`` to supply a specific gradient map for shading (:issue:`22727`) - :meth:`.Styler.clear` now clears :attr:`Styler.hidden_index` and :attr:`Styler.hidden_columns` as well (:issue:`40484`) - Added the method :meth:`.Styler.highlight_between` (:issue:`39821`) - Added the method :meth:`.Styler.highlight_quantile` (:issue:`40926`) - Added the method :meth:`.Styler.text_gradient` (:issue:`41098`) - - Added the method :meth:`.Styler.set_tooltips` allowing hover tooltips; this can be used enhance interactive displays (:issue:`21266`) - - Added the parameter ``escape`` to the method :meth:`.Styler.format` to escape data before it is passed to the formatter (:issue:`40388`) + - Added the method :meth:`.Styler.set_tooltips` to allow hover tooltips; this can be used enhance interactive displays (:issue:`21266`, :issue:`40284`) - Added the parameter ``precision`` to the method :meth:`.Styler.format` to control the display of floating point numbers (:issue:`40134`) - :class:`.Styler` rendered HTML output now follows the `w3 HTML Style Guide `_ (:issue:`39626`) - Many features of the :class:`.Styler` class are now either partially or fully usable on a DataFrame with a non-unique indexes or columns (:issue:`41143`) - One has greater control of the display through separate sparsification of the index or columns using the :ref:`new styler options `, which are also usable via :func:`option_context` (:issue:`41142`) -- Added the option ``sytler.max.elements`` to avoid browser overload when styling large DataFrames (:issue:`40712`) +- Added the option ``styler.render.max_elements`` to avoid browser overload when styling large DataFrames (:issue:`40712`) - Added the method :meth:`.Styler.to_latex` (:issue:`21673`) - Added the method :meth:`.Styler.to_html` (:issue:`13379`) @@ -163,6 +161,8 @@ a copy will no longer be made (:issue:`32960`). The default behavior when not passing ``copy`` will remain unchanged, i.e. a copy will be made. +.. _whatsnew_130.centered_datetimelike_rolling_window: + Centered datetime-like rolling windows ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From 9202b74fe9ad0179a40ee2ee85a844c756c75a42 Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Wed, 9 Jun 2021 00:08:27 -0400 Subject: [PATCH 8/9] Fix indentation --- doc/source/whatsnew/v1.3.0.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index 70a91b3454557..1ce2bcdd55cca 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -133,7 +133,7 @@ which has been revised and improved (:issue:`39720`, :issue:`39317`, :issue:`404 - :class:`.Styler` rendered HTML output now follows the `w3 HTML Style Guide `_ (:issue:`39626`) - Many features of the :class:`.Styler` class are now either partially or fully usable on a DataFrame with a non-unique indexes or columns (:issue:`41143`) - One has greater control of the display through separate sparsification of the index or columns using the :ref:`new styler options `, which are also usable via :func:`option_context` (:issue:`41142`) -- Added the option ``styler.render.max_elements`` to avoid browser overload when styling large DataFrames (:issue:`40712`) + - Added the option ``styler.render.max_elements`` to avoid browser overload when styling large DataFrames (:issue:`40712`) - Added the method :meth:`.Styler.to_latex` (:issue:`21673`) - Added the method :meth:`.Styler.to_html` (:issue:`13379`) From c510bd8ef531ec37e719fe4099e5b068b00b7b93 Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Wed, 9 Jun 2021 01:00:26 -0400 Subject: [PATCH 9/9] More indents --- doc/source/whatsnew/v1.3.0.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index 1ce2bcdd55cca..2945fc760e01a 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -120,11 +120,11 @@ which has been revised and improved (:issue:`39720`, :issue:`39317`, :issue:`404 - The method :meth:`.Styler.set_table_styles` can now accept more natural CSS language for arguments, such as ``'color:red;'`` instead of ``[('color', 'red')]`` (:issue:`39563`) - The methods :meth:`.Styler.highlight_null`, :meth:`.Styler.highlight_min`, and :meth:`.Styler.highlight_max` now allow custom CSS highlighting instead of the default background coloring (:issue:`40242`) -- :meth:`.Styler.apply` now accepts functions that return an ``ndarray`` when ``axis=None``, making it now consistent with the ``axis=0`` and ``axis=1`` behavior (:issue:`39359`) -- When incorrectly formatted CSS is given via :meth:`.Styler.apply` or :meth:`.Styler.applymap`, an error is now raised upon rendering (:issue:`39660`) + - :meth:`.Styler.apply` now accepts functions that return an ``ndarray`` when ``axis=None``, making it now consistent with the ``axis=0`` and ``axis=1`` behavior (:issue:`39359`) + - When incorrectly formatted CSS is given via :meth:`.Styler.apply` or :meth:`.Styler.applymap`, an error is now raised upon rendering (:issue:`39660`) - :meth:`.Styler.format` now accepts the keyword argument ``escape`` for optional HTML and LaTex escaping (:issue:`40388`, :issue:`41619`) -- :meth:`.Styler.background_gradient` has gained the argument ``gmap`` to supply a specific gradient map for shading (:issue:`22727`) -- :meth:`.Styler.clear` now clears :attr:`Styler.hidden_index` and :attr:`Styler.hidden_columns` as well (:issue:`40484`) + - :meth:`.Styler.background_gradient` has gained the argument ``gmap`` to supply a specific gradient map for shading (:issue:`22727`) + - :meth:`.Styler.clear` now clears :attr:`Styler.hidden_index` and :attr:`Styler.hidden_columns` as well (:issue:`40484`) - Added the method :meth:`.Styler.highlight_between` (:issue:`39821`) - Added the method :meth:`.Styler.highlight_quantile` (:issue:`40926`) - Added the method :meth:`.Styler.text_gradient` (:issue:`41098`)