Merge branch 'unifyai:main' into master

.pre-commit-config.yaml

@@ -32,12 +32,12 @@ repos:
     rev: v1.7.5
     hooks:
       - id: docformatter
-  - repo: https://github.com/pycqa/pydocstyle
-    rev: 6.3.0
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.1.8
     hooks:
-      - id: pydocstyle
-        # Exclude everything in frontends except __init__.py, and func_wrapper.py
-        exclude: 'ivy/functional/(frontends|backends)/(?!.*/func_wrapper\.py$).*(?!__init__\.py$)'
+      # Run the linter.
+      - id: ruff
+        args: [ --fix ]
   - repo: https://github.com/unifyai/lint-hook
     rev: a72ffb17562d919311653d7f593cb537d1245c19
     hooks:

available_configs.json

@@ -6,8 +6,5 @@
         "cp311-cp311-manylinux_2_17_x86_64",
         "cp310-cp310-macosx_arm64_darwin",
         "cp311-cp311-macosx_arm64_darwin"
-    ],
-    "engines": [
-        "cp310-cp310-manylinux_2_17_x86_64"
     ]
 }

binaries.json

@@ -65,31 +65,6 @@
                     "X.so"
                 ]
             }
-        ],
-        "engines": [
-            "ivy2xla.so",
-            {
-                "XLA": {
-                    "rust_api": {
-                        "python_frontend": [
-                            "activations.so",
-                            "creation.so",
-                            "elementwise.so",
-                            "general.so",
-                            "ivy2xla.so",
-                            "layers.so",
-                            "linear_algebra.so",
-                            "manipulation.so",
-                            "norms.so",
-                            "stateful_activations.so",
-                            "stateful_layers.so",
-                            "stateful_norms.so",
-                            "statistical.so",
-                            "xla_core.so"
-                        ]
-                    }
-                }
-            }
         ]
     }
 }

docs/overview/contributing/open_tasks.rst

@@ -20,6 +20,7 @@ We make no mention of task selection and allocation in the explanations below, w
 
 The tasks currently open are:
 
+#. Fixing Failing Tests
 #. Function Formatting
 #. Frontend APIs
 #. Ivy Experimental API
@@ -28,81 +29,25 @@ We try to explain these tasks as clearly as possible, but in cases where things
 
 Please always use the latest commit on GitHub when working on any of these tasks, **DO NOT** develop your code using the latest PyPI release of :code:`ivy`.
 
-Function Formatting
--------------------
-
-Currently, we have many ToDo list issues `open <https://github.com/unifyai/ivy/issues?q=is%3Aopen+is%3Aissue+label%3A%22Function+Reformatting%22+label%3AToDo>`_ for a general function formatting task, which is explained below.
-
-Each function in each submodule should be updated to follow the implementation instructions given in the `Deep Dive`_ section.
-The updates should be applied for the:
-
-#. ivy API
-#. all backend APIs
-#. array instance methods
-#. container instance methods
-#. array operators
-#. array reverse operators
-#. container operators
-#. container reverse operators
-
-The `Deep Dive`_ is an **essential** resource for learning how each of these functions/methods should be implemented.
-Before starting any contribution task, you should go through the `Deep Dive`_, and familiarize yourself with the content.
-
-At the time of writing, many of the functions are not implemented as they should be.
-You will need to make changes to the current implementations, but you do not need to address *all* sections of the `Deep Dive`_ in detail.
-Specifically, you **do not** need to address the following:
-
-#. Implement the hypothesis testing for the function
-#. Get the tests passing for your function, if they are failing before you start
-
-However, everything else covered in the `Deep Dive`_ must be addressed.
-Some common important tasks are:
-
-#. Remove all :code:`lambda` and direct bindings for the backend functions (in :code:`ivy.functional.backends`), with each function instead defined using :code:`def`.
-#. Implement the following if they don't exist but should do: :class:`ivy.Array` instance method, :class:`ivy.Container` instance method, :class:`ivy.Array` special method, :class:`ivy.Array` reverse special method, :class:`ivy.Container` special method, :class:`ivy.Container` reverse special method.
-#. Make sure that the aforementioned methods are added into the correct category-specific parent class, such as :class:`ivy.ArrayWithElementwise`, :class:`ivy.ContainerWithManipulation` etc.
-#. Correct all of the `Function Arguments <../deep_dive/function_arguments.rst>`_ and the type hints for every function **and** its *relevant methods*, including those you did not implement yourself.
-#. Add the correct `Docstrings <../deep_dive/docstrings.rst>`_ to every function **and** its *relevant methods*, including those you did not implement yourself.
-#. Add thorough `Docstring Examples <../deep_dive/docstring_examples.rst>`_ for every function **and** its *relevant methods* and ensure they pass the docstring tests.
-
-Formatting checklist
-~~~~~~~~~~~~~~~~~~~~
-
-After creating your Pull Request on github, you should then produce the checklist for the formatting task as follows:
-
-1. Add a comment with the following format: :code:`add_reformatting_checklist_<category_name>` on your PR, where *<category_name>* is the name of the category that the function belongs to.
-   An example of this is shown below.
-
-.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/contributing/open_tasks/checklist_generator.png?raw=true
-   :width: 420
-
-Using this formatting will then trigger our github automation bots to update your comment with the proper markdown text for the checklist.
-These updates might take a few moments to take effect, so please be patient 🙂.
-
-2. After adding the checklist to your PR, you should then modify this checklist with the status of each item according to the symbols(emojis) within the LEGEND section.
+Fixing Failing Tests
+--------------------
 
-.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/contributing/open_tasks/checklist_legend.png?raw=true
-   :width: 420
+In the Ivy repository, we've identified a range of functions and tests that currently fall short of our quality standards. The root of these issues is not always clear-cut. In some instances, the problem may lie within the functions themselves, while in others, it could be the tests, which were crafted by various contributors, that are at fault. This situation presents a unique opportunity for community involvement. We are actively inviting contributions from our community to help tackle these challenges.
 
-3. When all check items are marked as (✅, ⏩, or 🆗), you should request a review for your PR and we will start checking your implementation and marking the items as complete using the checkboxes next to them.
+How to Contribute
+~~~~~~~~~~~~~~~~~
 
-.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/contributing/open_tasks/checklist_checked.png?raw=true
-   :width: 420
+**Identifying Issues**
 
-4. In case you are stuck or need help with one of the checklist items, please add the 🆘 symbol next to the item on the checklist, and proceed to add a comment elaborating on your point of struggle with this item.
-The PR assignee will then see this comment and address your issues.
+To get started, visit our issues page: `Failing Tests <https://github.com/unifyai/ivy/issues?q=is%3Aissue+is%3Aopen+label%3A%22Failing+Test%22+label%3A%22ToDo%22>`_. Here, you will find a list of open issues labeled as "Failing Test" and "ToDo". These issues are categorised under various frameworks supported by our repository. We encourage you to select a framework you're comfortable with or interested in contributing to.
 
-.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/contributing/open_tasks/checklist_SOS.png?raw=true
-   :width: 420
+**Selecting a Test**
 
-**Notes**:
+Within each framework, tests are classified as either "Priority Open" or "Other Open." While we prioritize fixing the "Priority Open" tests, contributions towards any test, including those labeled "Other Open," are highly valuable. Each test issue is linked directly to the specific failing workflow. This linkage provides you with immediate access to the details of what exactly is failing and the context around it.
 
-1. It is important that the PR author is the one to add the checklist generating comment in order to ensure they will have access to edit and update it later.
-2. The checklist items' statuses should be manually updated by the PR author.
-   It does not automatically run any tests to update them!
-3. Do not edit the checklist text, only the emoji symbols. 😅
-4. Please refrain from using the checkboxes next to checklist items.
+**Making Your Contribution**
 
+After selecting a test to work on, please fork the repository and create a new branch for your fix. Ensure that your solution addresses the issue effectively and doesn't introduce new errors. Once you're confident in your fix, submit a pull request to the main repository. Our team will review your contribution, provide feedback if necessary, and then merge your changes once they meet our standards.
 
 Frontend APIs
 -------------
@@ -222,6 +167,83 @@ However, you can still use the checklist as a reference in cases where you do un
 2. Do not edit the checklist text, only the emoji symbols.
 3. Please refrain from using the checkboxes next to checklist items.
 
+
+Function Formatting
+-------------------
+
+Currently, we have many ToDo list issues `open <https://github.com/unifyai/ivy/issues?q=is%3Aopen+is%3Aissue+label%3A%22Function+Reformatting%22+label%3AToDo>`_ for a general function formatting task, which is explained below.
+
+Each function in each submodule should be updated to follow the implementation instructions given in the `Deep Dive`_ section.
+The updates should be applied for the:
+
+#. ivy API
+#. all backend APIs
+#. array instance methods
+#. container instance methods
+#. array operators
+#. array reverse operators
+#. container operators
+#. container reverse operators
+
+The `Deep Dive`_ is an **essential** resource for learning how each of these functions/methods should be implemented.
+Before starting any contribution task, you should go through the `Deep Dive`_, and familiarize yourself with the content.
+
+At the time of writing, many of the functions are not implemented as they should be.
+You will need to make changes to the current implementations, but you do not need to address *all* sections of the `Deep Dive`_ in detail.
+Specifically, you **do not** need to address the following:
+
+#. Implement the hypothesis testing for the function
+#. Get the tests passing for your function, if they are failing before you start
+
+However, everything else covered in the `Deep Dive`_ must be addressed.
+Some common important tasks are:
+
+#. Remove all :code:`lambda` and direct bindings for the backend functions (in :code:`ivy.functional.backends`), with each function instead defined using :code:`def`.
+#. Implement the following if they don't exist but should do: :class:`ivy.Array` instance method, :class:`ivy.Container` instance method, :class:`ivy.Array` special method, :class:`ivy.Array` reverse special method, :class:`ivy.Container` special method, :class:`ivy.Container` reverse special method.
+#. Make sure that the aforementioned methods are added into the correct category-specific parent class, such as :class:`ivy.ArrayWithElementwise`, :class:`ivy.ContainerWithManipulation` etc.
+#. Correct all of the `Function Arguments <../deep_dive/function_arguments.rst>`_ and the type hints for every function **and** its *relevant methods*, including those you did not implement yourself.
+#. Add the correct `Docstrings <../deep_dive/docstrings.rst>`_ to every function **and** its *relevant methods*, including those you did not implement yourself.
+#. Add thorough `Docstring Examples <../deep_dive/docstring_examples.rst>`_ for every function **and** its *relevant methods* and ensure they pass the docstring tests.
+
+Formatting checklist
+~~~~~~~~~~~~~~~~~~~~
+
+After creating your Pull Request on github, you should then produce the checklist for the formatting task as follows:
+
+1. Add a comment with the following format: :code:`add_reformatting_checklist_<category_name>` on your PR, where *<category_name>* is the name of the category that the function belongs to.
+   An example of this is shown below.
+
+.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/contributing/open_tasks/checklist_generator.png?raw=true
+   :width: 420
+
+Using this formatting will then trigger our github automation bots to update your comment with the proper markdown text for the checklist.
+These updates might take a few moments to take effect, so please be patient 🙂.
+
+2. After adding the checklist to your PR, you should then modify this checklist with the status of each item according to the symbols(emojis) within the LEGEND section.
+
+.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/contributing/open_tasks/checklist_legend.png?raw=true
+   :width: 420
+
+3. When all check items are marked as (✅, ⏩, or 🆗), you should request a review for your PR and we will start checking your implementation and marking the items as complete using the checkboxes next to them.
+
+.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/contributing/open_tasks/checklist_checked.png?raw=true
+   :width: 420
+
+4. In case you are stuck or need help with one of the checklist items, please add the 🆘 symbol next to the item on the checklist, and proceed to add a comment elaborating on your point of struggle with this item.
+The PR assignee will then see this comment and address your issues.
+
+.. image:: https://github.com/unifyai/unifyai.github.io/blob/main/img/externally_linked/contributing/open_tasks/checklist_SOS.png?raw=true
+   :width: 420
+
+**Notes**:
+
+1. It is important that the PR author is the one to add the checklist generating comment in order to ensure they will have access to edit and update it later.
+2. The checklist items' statuses should be manually updated by the PR author.
+   It does not automatically run any tests to update them!
+3. Do not edit the checklist text, only the emoji symbols. 😅
+4. Please refrain from using the checkboxes next to checklist items.
+
+
 Ivy Experimental API
 --------------------
 

docs/overview/contributing/the_basics.rst

@@ -52,7 +52,7 @@ We make extensive use of `ToDo list issues <https://github.com/unifyai/ivy/issue
 
 We have a clear process for contributors to engage with such ToDo lists:
 
-a. Find a task to work on which (i) is not marked as completed with a tick (ii) does not have an issue created and (iii) is not mentioned in the comments. Currently, there are three open tasks: :ref:`overview/contributing/open_tasks:Function Formatting`, :ref:`overview/contributing/open_tasks:Frontend APIs` and :ref:`overview/contributing/open_tasks:Ivy Experimental API`.
+a. Find a task to work on which (i) is not marked as completed with a tick (ii) does not have an issue created and (iii) is not mentioned in the comments. Currently, there are three open tasks: :ref:`overview/contributing/open_tasks:Fixing Failing Tests`, :ref:`overview/contributing/open_tasks:Function Formatting`, :ref:`overview/contributing/open_tasks:Frontend APIs` and :ref:`overview/contributing/open_tasks:Ivy Experimental API`.
 
 b. Create a new issue with the title being just the name of the sub-task you would like to work on.
 

install_dependencies.sh

@@ -1,6 +1,6 @@
 # This shell script is required by the doc-builder. Moving it might break
 # the doc-building pipeline
-
+pip install -e .
 pip install -r requirements/requirements.txt
 if [[ $(arch) == 'arm64' ]]; then
       brew install pandoc

ivy/compiler/compiler.py

@@ -94,7 +94,6 @@ def trace_graph(
     >>> print(time.time() - start)
     0.0001785755157470703
     """
-
     from ._compiler import trace_graph as _trace_graph
 
     return _trace_graph(
@@ -159,7 +158,6 @@ def transpile(
     -------
     Either a transpiled Graph or a non-initialized LazyGraph.
     """
-
     from ._compiler import transpile as _transpile
 
     return _transpile(

ivy/data_classes/container/base.py

@@ -1245,7 +1245,7 @@ def h5_file_size(h5_obj_or_filepath):
 
         size = 0
         batch_size = 0
-        for key, value in h5_obj.items():
+        for value in h5_obj.values():
             if isinstance(value, h5py.Group):
                 size_to_add, batch_size = ivy.Container.h5_file_size(value)
                 size += size_to_add
@@ -1285,7 +1285,7 @@ def shuffle_h5_file(h5_obj_or_filepath, seed_value=0):
         else:
             h5_obj = h5_obj_or_filepath
 
-        for key, value in h5_obj.items():
+        for value in h5_obj.values():
             if isinstance(value, h5py.Group):
                 ivy.Container.shuffle_h5_file(value, seed_value)
             elif isinstance(value, h5py.Dataset):
@@ -2160,7 +2160,7 @@ def cont_to_iterator_values(self, include_empty=False):
             Iterator for the container values.
 
         """
-        for key, value in self.items():
+        for value in self.values():
             if isinstance(value, ivy.Container) and (not include_empty or value):
                 # noinspection PyCompatibility
                 yield from value.cont_to_iterator_values(include_empty)

ivy/data_classes/container/experimental/linear_algebra.py

@@ -768,7 +768,7 @@ def multi_dot(
 
     @staticmethod
     def static_cond(
-        self: ivy.Container,
+        x: ivy.Container,
         /,
         *,
         key_chains: Optional[Union[List[str], Dict[str, str], ivy.Container]] = None,
@@ -784,7 +784,7 @@ def static_cond(
 
         Parameters
         ----------
-            self
+            x
                 container with input arrays.
             p
                 order of the norm of the matrix (see ivy.norm).
@@ -803,7 +803,7 @@ def static_cond(
         """
         return ContainerBase.cont_multi_map_in_function(
             "cond",
-            self,
+            x,
             p=p,
             out=out,
             key_chains=key_chains,

ivy/functional/backends/jax/data_type.py

@@ -267,7 +267,4 @@ def dtype_bits(dtype_in: Union[jnp.dtype, str, np.dtype], /) -> int:
 def is_native_dtype(dtype_in: Union[jnp.dtype, str], /) -> bool:
     if not ivy.is_hashable_dtype(dtype_in):
         return False
-    if dtype_in in ivy_dtype_dict:
-        return True
-    else:
-        return False
+    return dtype_in in ivy_dtype_dict

ivy/functional/backends/jax/experimental/losses.py

@@ -84,22 +84,22 @@ def _validate_poisson_nll_params(
     for parameter, name in zip([input, label], ["input", "label"]):
         if parameter.dtype not in allowed_dtypes:
             raise TypeError(
-                "The dtype of '%s' in poisson_nll_loss should be one of %s, but"
-                " received %s." % (name, allowed_dtypes, parameter.dtype)
+                f"The dtype of '{name}' in poisson_nll_loss should be one of"
+                f" {allowed_dtypes}, but received {parameter.dtype}."
             )
 
     # Validate epsilon
     if epsilon <= 0:
         raise ValueError(
             "The value of `epsilon` in poisson_nll_loss should be positive, but"
-            " received %f, which is not allowed" % epsilon
+            f" received {epsilon}, which is not allowed."
         )
 
     # Validate reduction
     if reduction not in ["sum", "mean", "none"]:
         raise ValueError(
             "The value of 'reduction' in poisson_nll_loss should be 'sum', 'mean' or"
-            " 'none', but received %s, which is not allowed." % reduction
+            f" 'none', but received {reduction}, which is not allowed."
         )
 
     # Validate shape

ivy/functional/backends/numpy/data_type.py

@@ -260,7 +260,4 @@ def dtype_bits(dtype_in: Union[np.dtype, str], /) -> int:
 def is_native_dtype(dtype_in: Union[np.dtype, str], /) -> bool:
     if not ivy.is_hashable_dtype(dtype_in):
         return False
-    if dtype_in in ivy_dtype_dict:
-        return True
-    else:
-        return False
+    return dtype_in in ivy_dtype_dict