diff options
author | Henrique Joaquim <h.joaquim@campus.fct.unl.pt> | 2024-02-01 10:00:47 +0000 |
---|---|---|
committer | GitHub <noreply@github.com> | 2024-02-01 10:00:47 +0000 |
commit | d4baba0284f80307b68206861a4b229a1908e79a (patch) | |
tree | 2702d3e6b0631da74d298899c5f3c8cb66454c15 | |
parent | 5fd11beaeefd95c2b52e223e40896c507829a5f4 (diff) |
[Feature] - Custom deprecation (#6005)
* custom deprecation
* custom deprecation
* using the new deprecation
* custom deprecation on the package builder
* remove comment
* ruff
* black
* static assets
* tests
* using parametrization instead
* test for deprecated endpoints (#6014)
* Deprecation warning on the reference docs (#6015)
* typo/fix
* bring back methods needed for markdown generation
* add deprecation warning to docs
* contributor docs for deprecating endpoints - tks @deeleeramone
* small changes on publishing procedure per @the-praxs
* moving the deprecation summary class to deprecation file instead
* explanation on class variables
* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
* Update openbb_platform/openbb/package/index.py
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
* deprecating on 4.3 instead @the-praxs
---------
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
13 files changed, 353 insertions, 27 deletions
diff --git a/build/pypi/openbb_platform/PUBLISH.md b/build/pypi/openbb_platform/PUBLISH.md index da81a603501..e844cd1b0fc 100644 --- a/build/pypi/openbb_platform/PUBLISH.md +++ b/build/pypi/openbb_platform/PUBLISH.md @@ -8,7 +8,7 @@ Publishing checklist: 2. Ensure all integration tests pass: `pytest openbb_platform -m integration` 3. Run `python -c "import openbb; openbb.build()"` to build the static assets. Make sure that only required extensions are installed. -> **Note** Run `python -c "import openbb"` after building the static to check that no additional static is being built. + > **Note** Run `python -c "import openbb"` after building the static to check that no additional static is being built. 4. Run the following commands for publishing the packages to PyPI: @@ -17,11 +17,13 @@ Publishing checklist: 1. For the core package run: `python build/pypi/openbb_platform/publish.py --core` 2. For the extension and provider packages run: `python build/pypi/openbb_platform/publish.py --extensions` - 3. For the `openbb` package, do the following + 3. For the `openbb` package - **which requires manual publishing** - do the following - Bump the dependency package versions - Re-build the static assets that are bundled with the package + - Run unit tests to validate the existence of deprecated endpoints - > Note that, in order for packages to pick up the latest versions of dependencies, it might be necessary to clear the local cache of the dependencies: + > [!TIP] + > Note that, in order for packages to pick up the latest versions of dependencies, it is advised to clear the local cache of the dependencies: > > We can do that with `pip cache purge` and `poetry cache clear pypi --all` > diff --git a/openbb_platform/core/openbb_core/app/deprecation.py b/openbb_platform/core/openbb_core/app/deprecation.py new file mode 100644 index 00000000000..f4e03c87687 --- /dev/null +++ b/openbb_platform/core/openbb_core/app/deprecation.py @@ -0,0 +1,62 @@ +""" +OpenBB-specific deprecation warnings. + +This implementation was inspired from Pydantic's specific warnings and modified to suit OpenBB's needs. +""" + +from typing import Optional, Tuple + + +class DeprecationSummary(str): + """A string subclass that can be used to store deprecation metadata.""" + + def __new__(cls, value, metadata): + """Create a new instance of the class.""" + obj = str.__new__(cls, value) + obj.metadata = metadata + return obj + + +class OpenBBDeprecationWarning(DeprecationWarning): + """ + A OpenBB specific deprecation warning. + + This warning is raised when using deprecated functionality in OpenBB. It provides information on when the + deprecation was introduced and the expected version in which the corresponding functionality will be removed. + + Attributes + ---------- + message: Description of the warning. + since: Version in what the deprecation was introduced. + expected_removal: Version in what the corresponding functionality expected to be removed. + """ + + # The choice to use class variables is based on the potential for extending the class in future developments. + # Example: launching Platform V5 and decide to create a subclimagine we areass named OpenBBDeprecatedSinceV4, + # which inherits from OpenBBDeprecationWarning. In this subclass, we would set since=4.X and expected_removal=5.0. + # It's important for these values to be defined at the class level, rather than just at the instance level, + # to ensure consistency and clarity in our deprecation warnings across the platform. + + message: str + since: Tuple[int, int] + expected_removal: Tuple[int, int] + + def __init__( + self, + message: str, + *args: object, + since: Tuple[int, int], + expected_removal: Optional[Tuple[int, int]] = None, + ) -> None: + super().__init__(message, *args) + self.message = message.rstrip(".") + self.since = since + self.expected_removal = expected_removal or (since[0] + 1, 0) + self.long_message = ( + f"{self.message}. Deprecated in OpenBB Platform V{self.since[0]}.{self.since[1]}" + f" to be removed in V{self.expected_removal[0]}.{self.expected_removal[1]}." + ) + + def __str__(self) -> str: + """Return the warning message.""" + return self.long_message diff --git a/openbb_platform/core/openbb_core/app/router.py b/openbb_platform/core/openbb_core/app/router.py index 2426f2f6da9..39e9f438227 100644 --- a/openbb_platform/core/openbb_core/app/router.py +++ b/openbb_platform/core/openbb_core/app/router.py @@ -24,6 +24,7 @@ from pydantic import BaseModel from pydantic.v1.validators import find_validators from typing_extensions import Annotated, ParamSpec, _AnnotatedAlias +from openbb_core.app.deprecation import DeprecationSummary, OpenBBDeprecationWarning from openbb_core.app.example_generator import ExampleGenerator from openbb_core.app.extension_loader import ExtensionLoader from openbb_core.app.model.abstract.warning import OpenBBWarning @@ -232,7 +233,6 @@ class Router: api_router = self._api_router model = kwargs.pop("model", "") - deprecation_message = kwargs.pop("deprecation_message", None) examples = kwargs.pop("examples", []) exclude_auto_examples = kwargs.pop("exclude_auto_examples", False) @@ -279,14 +279,13 @@ class Router: }, ) - # For custom deprecation messages + # For custom deprecation if kwargs.get("deprecated", False): - if deprecation_message: - kwargs["summary"] = deprecation_message - else: - kwargs["summary"] = ( - "This functionality will be deprecated in the future releases." - ) + deprecation: OpenBBDeprecationWarning = kwargs.pop("deprecation") + + kwargs["summary"] = DeprecationSummary( + deprecation.long_message, deprecation + ) api_router.add_api_route(**kwargs) diff --git a/openbb_platform/core/openbb_core/app/static/package_builder.py b/openbb_platform/core/openbb_core/app/static/package_builder.py index 7648ed89511..d9c6911c374 100644 --- a/openbb_platform/core/openbb_core/app/static/package_builder.py +++ b/openbb_platform/core/openbb_core/app/static/package_builder.py @@ -9,9 +9,11 @@ from inspect import Parameter, _empty, isclass, signature from json import dumps, load from pathlib import Path from typing import ( + Any, Callable, Dict, List, + Literal, Optional, OrderedDict, Set, @@ -20,6 +22,7 @@ from typing import ( TypeVar, Union, get_args, + get_origin, get_type_hints, ) @@ -35,7 +38,7 @@ from openbb_core.app.charting_service import ChartingService from openbb_core.app.extension_loader import ExtensionLoader, OpenBBGroups from openbb_core.app.model.custom_parameter import OpenBBCustomParameter from openbb_core.app.provider_interface import ProviderInterface -from openbb_core.app.router import RouterLoader +from openbb_core.app.router import CommandMap, RouterLoader from openbb_core.app.static.utils.console import Console from openbb_core.app.static.utils.linters import Linters from openbb_core.env import Env @@ -302,6 +305,8 @@ class ImportDefinition: for child_path in child_path_list: route = PathHandler.get_route(path=child_path, route_map=route_map) if route: + if route.deprecated: + hint_type_list.append(type(route.summary.metadata)) function_hint_type_list = cls.get_function_hint_type_list(func=route.endpoint) # type: ignore hint_type_list.extend(function_hint_type_list) @@ -331,10 +336,11 @@ class ImportDefinition: code += "\nimport typing" code += "\nfrom typing import List, Dict, Union, Optional, Literal" code += "\nfrom annotated_types import Ge, Le, Gt, Lt" + code += "\nfrom warnings import warn, simplefilter" if sys.version_info < (3, 9): code += "\nimport typing_extensions" else: - code += "\nfrom typing_extensions import Annotated" + code += "\nfrom typing_extensions import Annotated, deprecated" code += "\nfrom openbb_core.app.utils import df_to_basemodel" code += "\nfrom openbb_core.app.static.utils.decorators import validate\n" code += "\nfrom openbb_core.app.static.utils.filters import filter_inputs\n" @@ -347,6 +353,15 @@ class ImportDefinition: module_list = list(set(module_list)) module_list.sort() + specific_imports = { + hint_type.__module__: hint_type.__name__ + for hint_type in hint_type_list + if getattr(hint_type, "__name__", None) is not None + } + code += "\n" + for module, name in specific_imports.items(): + code += f"from {module} import {name}\n" + code += "\n" for module in module_list: code += f"import {module}\n" @@ -637,6 +652,7 @@ class MethodDefinition: func_name: str, formatted_params: OrderedDict[str, Parameter], return_type: type, + path: str, model_name: Optional[str] = None, ) -> str: """Build the command method signature.""" @@ -651,7 +667,20 @@ class MethodDefinition: if "pandas.DataFrame" in func_params else "" ) - code = f"\n @validate{args}" + + msg = "" + if MethodDefinition.is_deprecated_function(path): + deprecation_message = MethodDefinition.get_deprecation_message(path) + deprecation_type_class = type( + deprecation_message.metadata # type: ignore + ).__name__ + + msg = "\n @deprecated(" + msg += f'\n "{deprecation_message}",' + msg += f"\n category={deprecation_type_class}," + msg += "\n )" + + code = f"\n @validate{args}{msg}" code += f"\n def {func_name}(" code += f"\n self,\n {func_params}\n ) -> {func_returns}:\n" @@ -698,7 +727,7 @@ class MethodDefinition: if MethodDefinition.is_deprecated_function(path): deprecation_message = MethodDefinition.get_deprecation_message(path) - code += " from warnings import warn, simplefilter; simplefilter('always', DeprecationWarning)\n" + code += " simplefilter('always', DeprecationWarning)\n" code += f""" warn("{deprecation_message}", category=DeprecationWarning, stacklevel=2)\n\n""" code += " return self._run(\n" @@ -748,6 +777,7 @@ class MethodDefinition: func_name=func_name, formatted_params=formatted_params, return_type=sig.return_annotation, + path=path, model_name=model_name, ) code += cls.build_command_method_doc( @@ -957,6 +987,114 @@ class DocstringGenerator: return doc return doc + @staticmethod + def get_model_standard_params(param_fields: Dict[str, FieldInfo]) -> Dict[str, Any]: + """Get the test params for the fetcher based on the required standard params.""" + test_params: Dict[str, Any] = {} + for field_name, field in param_fields.items(): + if field.default and field.default is not PydanticUndefined: + test_params[field_name] = field.default + elif field.default and field.default is PydanticUndefined: + example_dict = { + "symbol": "AAPL", + "symbols": "AAPL,MSFT", + "start_date": "2023-01-01", + "end_date": "2023-06-06", + "country": "Portugal", + "date": "2023-01-01", + "countries": ["portugal", "spain"], + } + if field_name in example_dict: + test_params[field_name] = example_dict[field_name] + elif field.annotation == str: + test_params[field_name] = "TEST_STRING" + elif field.annotation == int: + test_params[field_name] = 1 + elif field.annotation == float: + test_params[field_name] = 1.0 + elif field.annotation == bool: + test_params[field_name] = True + elif get_origin(field.annotation) is Literal: # type: ignore + option = field.annotation.__args__[0] # type: ignore + if isinstance(option, str): + test_params[field_name] = f'"{option}"' + else: + test_params[field_name] = option + + return test_params + + @staticmethod + def get_full_command_name(route: str) -> str: + """Get the full command name.""" + cmd_parts = route.split("/") + del cmd_parts[0] + + menu = cmd_parts[0] + command = cmd_parts[-1] + sub_menus = cmd_parts[1:-1] + + sub_menu_str_cmd = f".{'.'.join(sub_menus)}" if sub_menus else "" + + full_command = f"{menu}{sub_menu_str_cmd}.{command}" + + return full_command + + @classmethod + def generate_example( + cls, + model_name: str, + standard_params: Dict[str, FieldInfo], + ) -> str: + """Generate the example for the command.""" + # find the model router here + cm = CommandMap() + commands_model = cm.commands_model + route = [k for k, v in commands_model.items() if v == model_name] + + if not route: + return "" + + full_command_name = cls.get_full_command_name(route=route[0]) + example_params = cls.get_model_standard_params(param_fields=standard_params) + + # Edge cases (might find more) + if "crypto" in route[0] and "symbol" in example_params: + example_params["symbol"] = "BTCUSD" + elif "currency" in route[0] and "symbol" in example_params: + example_params["symbol"] = "EURUSD" + elif ( + "index" in route[0] + and "european" not in route[0] + and "symbol" in example_params + ): + example_params["symbol"] = "SPX" + elif ( + "index" in route[0] + and "european" in route[0] + and "symbol" in example_params + ): + example_params["symbol"] = "BUKBUS" + elif ( + "futures" in route[0] and "curve" in route[0] and "symbol" in example_params + ): + example_params["symbol"] = "VX" + elif "futures" in route[0] and "symbol" in example_params: + example_params["symbol"] = "ES" + + example = "\n Example\n -------\n" + example += " >>> from openbb import obb\n" + example += f" >>> obb.{full_command_name}(" + for param_name, param_value in example_params.items(): + if isinstance(param_value, str): + param_value = f'"{param_value}"' # noqa: PLW2901 + example += f"{param_name}={param_value}, " + if example_params: + example = example[:-2] + ")\n" + else: + example += ")\n" + + return example + class PathHandler: """Handle the paths for the Platform.""" diff --git a/openbb_platform/core/tests/app/static/test_package_builder.py b/openbb_platform/core/tests/app/static/test_package_builder.py index e90d93ddca0..44e7ab698d5 100644 --- a/openbb_platform/core/tests/app/static/test_package_builder.py +++ b/openbb_platform/core/tests/app/static/test_package_builder.py @@ -261,8 +261,10 @@ def test_build_func_returns(method_definition, return_type, expected_output): assert output == expected_output -def test_build_command_method_signature(method_definition): +@patch("openbb_core.app.static.package_builder.MethodDefinition") +def test_build_command_method_signature(mock_method_definitions, method_definition): """Test build command method signature.""" + mock_method_definitions.is_deprecated_function.return_value = False formatted_params = { "param1": Parameter("NoneType", kind=Parameter.POSITIONAL_OR_KEYWORD), "param2": Parameter("int", kind=Parameter.POSITIONAL_OR_KEYWORD), @@ -272,10 +274,31 @@ def test_build_command_method_signature(method_definition): func_name="test_func", formatted_params=formatted_params, return_type=return_type, + path="test_path", ) assert output +@patch("openbb_core.app.static.package_builder.MethodDefinition") +def test_build_command_method_signature_deprecated( + mock_method_definitions, method_definition +): + """Test build command method signature.""" + mock_method_definitions.is_deprecated_function.return_value = True + formatted_params = { + "param1": Parameter("NoneType", kind=Parameter.POSITIONAL_OR_KEYWORD), + "param2": Parameter("int", kind=Parameter.POSITIONAL_OR_KEYWORD), + } + return_type = int + output = method_definition.build_command_method_signature( + func_name="test_func", + formatted_params=formatted_params, + return_type=return_type, + path="test_path", + ) + assert "@deprecated" in output + + def test_build_command_method_doc(method_definition): """Test build command method doc.""" diff --git a/openbb_platform/core/tests/app/test_deprecation.py b/openbb_platform/core/tests/app/test_deprecation.py new file mode 100644 index 00000000000..3abec9e272d --- /dev/null +++ b/openbb_platform/core/tests/app/test_deprecation.py @@ -0,0 +1,28 @@ +import unittest + +from openbb_core.app.static.package_builder import PathHandler +from openbb_core.app.version import VERSION + + +def get_major_minor(version): + parts = version.split(".") + return (int(parts[0]), int(parts[1])) + + +class DeprecatedCommandsTest(unittest.TestCase): + """Test deprecated commands.""" + + def test_deprecated_commands(self): + """Test deprecated commands.""" + current_major_minor = get_major_minor(VERSION) + route_map = PathHandler.build_route_map() + + for path, route in route_map.items(): + with self.subTest(i=path): + if getattr(route, "deprecated", False): + deprecation_message = getattr(route, "summary", "") + obb_deprecation_warning = deprecation_message.metadata + + assert ( + obb_deprecation_warning.expected_removal != current_major_minor + ), f"The expected removal version of `{path}` matches the current version, please remove it." diff --git a/openbb_platform/extensions/index/openbb_index/index_router.py b/openbb_platform/extensions/index/openbb_index/index_router.py index 0769c6f5878..afcd0cd8d3f 100644 --- a/openbb_platform/extensions/index/openbb_index/index_router.py +++ b/openbb_platform/extensions/index/openbb_index/index_router.py @@ -1,5 +1,6 @@ """Index Router.""" +from openbb_core.app.deprecation import OpenBBDeprecationWarning from openbb_core.app.model.command_context import CommandContext from openbb_core.app.model.obbject import OBBject from openbb_core.app.provider_interface import ( @@ -22,7 +23,11 @@ router.include_router(price_router) @router.command( model="MarketIndices", deprecated=True, - deprecation_message="This endpoint will be deprecated in the future releases. Use '/index/price/historical' instead.", + deprecation=OpenBBDeprecationWarning( + message="This endpoint is deprecated; use `/index/price/historical` instead.", + since=(4, 1), + expected_removal=(4, 3), + ), ) async def market( cc: CommandContext, diff --git a/openbb_platform/openbb/package/extension_map.json b/openbb_platform/openbb/package/extension_map.json index cb31119c7f8..d2deb1107cc 100644 --- a/openbb_platform/openbb/package/extension_map.json +++ b/openbb_platform/openbb/package/extension_map.json @@ -24,4 +24,4 @@ "tradingeconomics@1.1.1", "yfinance@1.1.1" ] -} +}
\ No newline at end of file diff --git a/openbb_platform/openbb/package/index.py b/openbb_platform/openbb/package/index.py index ddee5141cbb..1caae2443c9 100644 --- a/openbb_platform/openbb/package/index.py +++ b/openbb_platform/openbb/package/index.py @@ -2,13 +2,15 @@ import datetime from typing import List, Literal, Optional, Union +from warnings import simplefilter, warn +from openbb_core.app.deprecation import OpenBBDeprecatedSince41 from openbb_core.app.model.custom_parameter import OpenBBCustomParameter from openbb_core.app.model.obbject import OBBject from openbb_core.app.static.container import Container from openbb_core.app.static.utils.decorators import validate from openbb_core.app.static.utils.filters import filter_inputs -from typing_extensions import Annotated +from typing_extensions import Annotated, deprecated class ROUTER_index(Container): @@ -155,6 +157,10 @@ class ROUTER_index(Container): ) @validate + @deprecated( + "This endpoint is deprecated; use `/index/price/historical` instead. Deprecated in OpenBB Platform V4.1 to be removed in V4.5.", + category=OpenBBDeprecatedSince41, + ) def market( self, symbol: Annotated[ @@ -266,11 +272,9 @@ class ROUTER_index(Container): >>> obb.index.market(symbol="SPX") """ # noqa: E501 - from warnings import simplefilter, warn - simplefilter("always", DeprecationWarning) warn( - "This endpoint will be deprecated in the future releases. Use '/index/price/historical' instead.", + "This endpoint is deprecated since v4.1 and will be removed in v4.3; Use `/index/price/historical` instead.", category=DeprecationWarning, stacklevel=2, ) diff --git a/openbb_platform/openbb/package/module_map.json b/openbb_platform/openbb/package/module_map.json index 3c43bc00321..a14f4bc4032 100644 --- a/openbb_platform/openbb/package/module_map.json +++ b/openbb_platform/openbb/package/module_map.json @@ -140,4 +140,4 @@ "regulators_sec_schema_files": "/regulators/sec/schema_files", "regulators_sec_sic_search": "/regulators/sec/sic_search", "regulators_sec_symbol_map": "/regulators/sec/symbol_map" -} +}
\ No newline at end of file diff --git a/website/content/platform/development/contributor-guidelines/deprecating_endpoints.md b/website/content/platform/development/contributor-guidelines/deprecating_endpoints.md new file mode 100644 index 00000000000..b38473be2d3 --- /dev/null +++ b/website/content/platform/development/contributor-guidelines/deprecating_endpoints.md @@ -0,0 +1,57 @@ +--- +title: Deprecating Endpoints +sidebar_position: 5 +description: This guide provides detailed instructions on how to deprecate an endpoint in the OpenBB Platform. +keywords: +- OpenBB community +- OpenBB Platform +- Custom commands +- API +- Python Interface +- Deprecation +- Deprecated +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + +<HeadTitle title="Deprecating Endpoints - Contributor Guidelines - Development | OpenBB Platform Docs" /> + +Deprecating commands is essential to maintaining the OpenBB Platform. This guide outlines the process for deprecating an endpoint. + +## Deprecating an endpoint + +1. Add the new endpoint that will replace the deprecated one. + +2. Add the deprecation warning + + Navigate to the **router** where the endpoint to be deprecated exists. Set the `deprecated` flag to `True` and add `deprecation=OpenBBDeprecationWarning(…)` argument to the decorator. Refer to the example below: + + ```python + + ... + from openbb_core.app.deprecation import OpenBBDeprecationWarning + ... + + @router.command( + model="MarketIndices", + deprecated=True, + deprecation=OpenBBDeprecationWarning( + message="This endpoint is deprecated; use `/index/price/historical` instead.", + since=(4, 1), + expected_removal=(4, 5), + ), + ) + async def market( + cc: CommandContext, + provider_choices: ProviderChoices, + standard_params: StandardParams, + extra_params: ExtraParams, + ) -> OBBject[BaseModel]: |