Add OAuth2 docs and Building Python API lib (#342)

* Remove log file

* Add OAuth2 docs and API lib tutorial

* Update link

* Fix typo

docs/api_lib_auth.md

@@ -0,0 +1,263 @@
+---
+title: "Python Library: Authentication"
+sidebar_label: Authentication
+---
+
+The Authentication part of your library is responsible for acquiring authentication and for making authenticated requests. It should not be aware of what is in the requests.
+
+Authentication comes in many forms, but it generally boils down to that each request is accompanied by an `authorization` header which contains an access token. The access token is generally a string of random numbers/letters.
+
+Your library should be able to acquire the authentication tokens, update them if necessary and use the authentication to make requests. It should not offer features to store the authentication data.
+
+Because authentication is going to be stored by the developer, it is important that you return the authentication to the developer in a format that can be JSON serializable. A `dict` with primitive types (`str`, `float`, `int`) is recommended.
+
+If your API can be served from multiple locations, your authentication class should allow the developer to pass in the location of the API.
+
+## Async example
+
+Python allows developers to write code that is either synchronous or asynchronous (via `asyncio`). Home Assistant is written in async, but is able to work with synchronous libraries too. We prefer async libraries.
+
+If you are writing a library in async, we recommend that you use `aiohttp`. It's a modern and mature HTTP library and is easy to use.
+
+```python
+from aiohttp import ClientSession, ClientResponse
+
+
+class Auth:
+    """Class to make authenticated requests."""
+
+    def __init__(self, websession: ClientSession, host: str, , access_token: str):
+        """Initialize the auth."""
+        self.websession = websession
+        self.host = host
+        self.access_token = access_token
+
+    async def request(self, method: str, path: str, **kwargs) -> ClientResponse:
+        """Make a request."""
+        headers = kwargs.get('headers')
+
+        if headers is None:
+            headers = {}
+        else:
+            headers = dict(headers)
+
+        headers["authorization"] = self.access_token
+
+        return await self.websession.request(
+            method,
+            f"{self.host}/{path}",
+            **kwargs,
+            headers=headers,
+        )
+```
+
+To use this class, you will need to create an aiohttp ClientSession and pass it together with the API info to the constructor.
+
+```python
+import asyncio
+import aiohttp
+
+from my_package import Auth
+
+
+async def main():
+    async with aiohttp.ClientSession() as session:
+        auth = Auth(session, "http://example.com/api", "secret_access_token")
+
+        # This will fetch data from http://example.com/api/lights
+        resp = await auth.request('get', 'lights')
+        print("HTTP response status code", resp.status)
+        print("HTTP response JSON content", await resp.json())
+
+asyncio.run(main())
+```
+
+## Sync example
+
+```python
+import requests
+
+
+class Auth:
+    """Class to make authenticated requests."""
+
+    def __init__(self, host: str, , access_token: str):
+        """Initialize the auth."""
+        self.host = host
+        self.access_token = access_token
+
+    async def request(self, method: str, path: str, **kwargs) -> requests.Response:
+        """Make a request."""
+        headers = kwargs.get('headers')
+
+        if headers is None:
+            headers = {}
+        else:
+            headers = dict(headers)
+
+        headers["authorization"] = self.access_token
+
+        return requests.request(
+            method,
+            f"{self.host}/{path}",
+            **kwargs,
+            headers=headers,
+        )
+```
+
+To use this class, construct the class with the API info.
+
+```python
+from my_package import Auth
+
+
+auth = Auth("http://example.com/api", "secret_access_token")
+
+# This will fetch data from http://example.com/api/lights
+resp = auth.request('get', 'lights')
+print("HTTP response status code", resp.status_code)
+print("HTTP response JSON content", resp.json())
+```
+
+## OAuth2
+
+OAuth2 is a [standardized version](https://tools.ietf.org/html/rfc6749) of an authentication schema leveraging refresh and access tokens. The access token expires within a short period of time after being issued. The refresh token can be used to acquire new access tokens.
+
+Refreshing access tokens relies on a client ID and secret, which might be held by an external service. We need to structure the authentication class to be able to allow the developer to implement their own token refresh logic.
+
+Home Assistant ships with the Home Assistant Cloud Account Linking service, a free cloud service to allow users to quickly connect accounts using OAuth2. Home Assistant has easy to use tools built-in to allow users to configure OAuth2-based integrations. For more info, [read here](config_entries_config_flow_handler.md#configuration-via-oauth2). These built-in tools work best if your library is implemented like the examples below.
+
+### Async example
+
+```python
+from abc import ABC, abstractmethod
+
+
+class AbstractAuth(ABC):
+    """Abstract class to make authenticated requests."""
+
+    def __init__(self, websession: ClientSession, host: str):
+        """Initialize the auth."""
+        self.websession = websession
+        self.host = host
+
+    @abstractmethod
+    async def async_get_access_token(self) -> str:
+        """Return a valid access token."""
+
+    async def request(self, method, url, **kwargs) -> ClientResponse:
+        """Make a request."""
+        headers = kwargs.get('headers')
+
+        if headers is None:
+            headers = {}
+        else:
+            headers = dict(headers)
+
+        access_token = await self.async_get_access_token()
+        headers["authorization"] = f"Bearer {access_token}"
+
+        return await self.websession.request(
+            method,
+            f"{self.host}/{url}",
+            **kwargs,
+            headers=headers,
+        )
+```
+
+Now the developer that is using your library will have to implement the abstract method for getting the access token. Let's assume that the developer has their own token manager class.
+
+```python
+from my_package import AbstractAuth
+
+
+class Auth(AbstractAuth):
+
+    def __init__(self, websession: ClientSession, host: str, token_manager):
+        """Initialize the auth."""
+        super().__init__(websession, host)
+        self.token_manager = token_manager
+
+    async def async_get_access_token(self) -> str:
+        """Return a valid access token."""
+        if self.token_manager.is_token_valid():
+            return self.token_manager.access_token
+
+        await self.token_manager.fetch_access_token()
+        await self.token_manager.save_access_token()
+
+        return self.token_manager.access_token
+```
+
+### Sync example
+
+If you are using `requests`, we recommend that you use the `requests_oauthlib` package. Below is an example that works with a local client ID and secret but also allows outsourcing token fetching to Home Assistant.
+
+```python
+from typing import Optional, Union, Callable, Dict
+
+from requests import Response
+from requests_oauthlib import OAuth2Session
+from oauthlib.oauth2 import TokenExpiredError
+
+
+class Auth:
+    def __init__(
+        self,
+        host: str,
+        token: Optional[Dict[str, str]] = None,
+        client_id: str = None,
+        client_secret: str = None,
+        token_updater: Optional[Callable[[str], None]] = None,
+    ):
+        self.host = host
+        self.client_id = client_id
+        self.client_secret = client_secret
+        self.token_updater = token_updater
+
+        self._oauth = OAuth2Session(
+            client_id=client_id,
+            client_secret=client_secret,
+            token=token,
+            token_updater=token_updater,
+        )
+
+    def refresh_tokens(self) -> Dict[str, Union[str, int]]:
+        """Refresh and return new tokens."""
+        token = self._oauth.refresh_token(f"{self.host}/auth/token")
+
+        if self.token_updater is not None:
+            self.token_updater(token)
+
+        return token
+
+    def request(self, method: str, path: str, **kwargs) -> Response:
+        """Make a request.
+
+        We don't use the built-in token refresh mechanism of OAuth2 session because
+        we want to allow overriding the token refresh logic.
+        """
+        url = f"{self.host}/{path}"
+        try:
+            return getattr(self._oauth, method)(url, **kwargs)
+        except TokenExpiredError:
+            self._oauth.token = self.refresh_tokens()
+
+            return getattr(self._oauth, method)(url, **kwargs)
+```
+
+A developer will now be able to override the refresh token function to route it via their own external service.
+
+```python
+from my_package import AbstractAuth
+
+
+class Auth(AbstractAuth):
+
+    def refresh_tokens(self) -> Dict[str, Union[str, int]]:
+        """Refresh and return new tokens."""
+        self.token_manager.fetch_access_token()
+        self.token_manager.save_access_token()
+
+        return self.token_manager.access_token
+```

docs/api_lib_data_models.md

@@ -0,0 +1,146 @@
+---
+title: "Python Library: Modelling Data"
+sidebar_label: Modelling Data
+---
+
+Now that we have authentication going, we can start making authenticated requests and fetch data!
+
+When modelling the data, it is important that we expose the data from the API in the same structure as that the API offers it. Some API designs might not make a lot of sense or contain typos. It is important that we still represent them in our objects. This makes it easy for developers using your library to follow the API documentation and know how it will work in your library.
+
+API libraries should try to do as little as possible. So it is okay to represent data structures as classes, but you should not transform data from one value into another. For example, you should not implement conversion between Celsius and Fahrenheit temperatures. This involves making decisions on precisions of results and should therefore be left to the developer using the library.
+
+For this example we're going to model an async library for a Rest API named ExampleHub that has two endpoints:
+
+- get `/light/<id>`: query the information of a single light.
+
+  ```json
+  {
+    "id": 1234,
+    "name": "Example Light",
+    "is_on": true
+  }
+  ```
+
+- post `/light/<id>`: control the light. Example JSON to send: `{ "is_on": false }`. Responds with the new state of the light.
+
+- get `/lights`: return a list of all lights
+  ```json
+  [
+    {
+      "id": 1234,
+      "name": "Example Light",
+      "is_on": true
+    },
+    {
+      "id": 5678,
+      "name": "Example Light 2",
+      "is_on": false
+    }
+  ]
+  ```
+
+As this API represents lights, we're first going to create a class to represent a light.
+
+```python
+from .auth import Auth
+
+
+class Light:
+    """Class that represents a Light object in the ExampleHub API."""
+
+    def __init__(self, raw_data: dict, auth: Auth):
+        """Initialize a light object."""
+        self.raw_data = raw_data
+        self.auth = auth
+
+    # Note: each property name maps the name in the returned data
+
+    @property
+    def id(self) -> int:
+        """Return the ID of the light."""
+        return self.raw_data['id']
+
+    @property
+    def name(self) -> str:
+        """Return the name of the light."""
+        return self.raw_data['name']
+
+    @property
+    def is_on(self) -> bool:
+        """Return if the light is on."""
+        return self.raw_data['id']
+
+    async def async_control(self, is_on: bool):
+        """Control the light."""
+        resp = await self.auth.request('post', f'light/{self.id}', json={
+          'is_on': is_on
+        })
+        resp.raise_for_status()
+        self.raw_data = await resp.json()
+
+    async def async_update(self):
+        """Update the light data."""
+        resp = await self.auth.request('get', f'light/{self.id}')
+        resp.raise_for_status()
+        self.raw_data = await resp.json()
+```
+
+Now that we have a light class, we can model the root of the API, which provides the entry points into the data.
+
+```python
+from typing import List
+
+from .auth import Auth
+from .light import Light
+
+
+class ExampleHubAPI:
+    """Class to communicate with the ExampleHub API."""
+
+    def __init__(self, auth: Auth):
+        """Initialize the API and store the auth so we can make requests."""
+        self.auth = auth
+
+    async def async_get_lights(self) -> List[Light]:
+        """Return the lights."""
+        resp = await self.auth.request('get', 'lights')
+        resp.raise_for_status()
+        return [
+          Light(light_data, self.auth)
+          for light_data in await resp.json()
+        ]
+
+    async def async_get_light(self, light_id) -> Light:
+        """Return the lights."""
+        resp = await self.auth.request('get', f'light/{light_id}')
+        resp.raise_for_status()
+        return Light(await resp.json(), self.auth)
+```
+
+With these two files in place, we can now control our lights like this:
+
+```python
+import asyncio
+import aiohttp
+
+from my_package import Auth, ExampleHubAPI
+
+
+async def main():
+    async with aiohttp.ClientSession() as session:
+        auth = Auth(session, "http://example.com/api", "secret_access_token")
+        api = ExampleHubAPI(auth)
+
+        lights = await api.async_get_lights()
+
+        # Print light states
+        for light in lights:
+            print(f"The light {light.name} is {light.is_on}")
+
+        # Control a light.
+        light = lights[0]
+        await light.async_control(not light.is_on)
+
+
+asyncio.run(main())
+```

docs/api_lib_index.md

@@ -0,0 +1,44 @@
+---
+title: "Building a Python library for an API"
+sidebar_label: "Introduction"
+---
+
+One of the foundational rules of Home Assistant is that we do not include any protocol specific code. Instead, this code should be put into a standalone Python library and published to PyPI. This guide will describe how to get started with this!
+
+For this guide we're going to assume that we're building a library for a Rest API that is accessible over HTTP and returning data structured as JSON objects. This is the most common type of API that we see. These APIs can either be accessible on the device itself, or in the cloud.
+
+This guide is not a perfect fit for every API. You might have to tweak the examples.
+
+> If you are a manufacturer designing a new API for your product, [please read about the best type of API to add to your products here](https://www.home-assistant.io/blog/2016/02/12/classifying-the-internet-of-things/#local-device-pushing-new-state).
+
+HTTP API requests consist of four different parts:
+
+- The URL. This is the path that we fetch data from. With a Rest API the URL will uniquely identify the resource. Examples of urls are `http://example.com/api/lights` and `http://example.com/api/light/1234`.
+- The HTTP method. This defines what we want from the API. The most common ones are:
+  - `GET` for when we want to get information like the state of a light
+  - `POST` for if we want something to be done (ie turn on a light)
+- The body. This is the data that we sent to the server to identify what needs to be done. This is how we send the command in the case of a `POST` request.
+- The headers. This contains metadata to describe your request. This will used to attach the authorization to the request.
+
+## Structuring the library
+
+Our library will consist of two different parts:
+
+- **Authentication:** Responsible for making authenticated HTTP requests to the API endpoint and returning the results. This is the only piece of code that will actually interact with the API.
+- **Data models:** Represent the data and offer commands to interact with the data.
+
+## Trying your library inside Home Assistant
+
+You will need to run an editable version of your library if you want to try your library in Home Assistant before it is publised to PyPI.
+
+Do so by going to your Home Assistant development environment, activating the virtual environment and typing:
+
+```
+pip3 install -e ../my_lib_folder
+```
+
+Now run Home Assistant without installing dependencies from PyPI to avoid overriding your package.
+
+```
+hass --skip-pip
+```

docs/config_entries_config_flow_handler.md

@@ -69,33 +69,15 @@ If your integration is discoverable without requiring any authentication, you'll
 - Support all manifest-based discovery protocols.
 - Limit to only 1 config entry. It is up to the config entry to discover all available devices.
 
-```python
-"""Config flow for LIFX."""
-from homeassistant.helpers import config_entry_flow
-from homeassistant import config_entries
+To get started, run `python3 -m script.scaffold config_flow_discovery` and follow the instructions. This will create all the boilerplate necessary to configure your integration using discovery.
 
-import aiolifx
+## Configuration via OAuth2
 
-from .const import DOMAIN
+Home Assistant has built-in support for integrations that offer account linking using [the OAuth2 authorization framework](https://tools.ietf.org/html/rfc6749). To be able to leverage this, you will need to structure your Python API library in a way that allows Home Assistant to be responsible for refreshing tokens. See our [API library guide](api_lib_index.md) on how to do this.
 
+The built-in OAuth2 support works out of the box with locally configured client ID / secret and with the Home Assistant Cloud Account Linking service. This service allows users to link their account with a centrally managed client ID/secret. If you want your integration to be part of this service, reach out to us at [hello@home-assistant.io](mailto:hello@home-assistant.io).
 
-async def _async_has_devices(hass):
-    """Return if there are devices that can be discovered."""
-    lifx_ip_addresses = await aiolifx.LifxScan(hass.loop).scan()
-    return len(lifx_ip_addresses) > 0
-
-
-config_entry_flow.register_discovery_flow(
-    # Domain of your integration
-    DOMAIN,
-    # Title of the created config entry
-    'LIFX',
-    # async method that returns a boolean if devices/services are found
-    _async_has_devices,
-    # Connection class of the integration
-    config_entries.CONN_CLASS_LOCAL_POLL
-)
-```
+To get started, run `python3 -m script.scaffold config_flow_oauth2` and follow the instructions. This will create all the boilerplate necessary to configure your integration using OAuth2.
 
 ## Translations
 

website/i18n/en.json

@@ -5,6 +5,18 @@
     "previous": "Previous",
     "tagline": "All you need to start developing for Home Assistant",
     "docs": {
+      "api_lib_auth": {
+        "title": "Python Library: Authentication",
+        "sidebar_label": "Authentication"
+      },
+      "api_lib_data_models": {
+        "title": "Python Library: Modelling Data",
+        "sidebar_label": "Modelling Data"
+      },
+      "api_lib_index": {
+        "title": "Building a Python library for an API",
+        "sidebar_label": "Introduction"
+      },
       "app_integration_index": {
         "title": "Native App Integration",
         "sidebar_label": "Introduction"
@@ -74,6 +86,10 @@
       "auth_permissions": {
         "title": "Permissions"
       },
+      "config_entries_config_flow_handler_oauth2": {
+        "title": "Integration Configuration with OAuth2",
+        "sidebar_label": "Configuration with OAuth2"
+      },
       "config_entries_config_flow_handler": {
         "title": "Integration Configuration",
         "sidebar_label": "Configuration"
@@ -1720,6 +1736,7 @@
       "Documentation": "Documentation",
       "Intents": "Intents",
       "Native App Integration": "Native App Integration",
+      "Building a Python library": "Building a Python library",
       "asyncio": "asyncio",
       "Hass.io": "Hass.io",
       "Hass.io Add-Ons": "Hass.io Add-Ons",

website/sidebars.json

@@ -129,6 +129,11 @@
       "app_integration_notifications",
       "app_integration_webview"
     ],
+    "Building a Python library": [
+      "api_lib_index",
+      "api_lib_auth",
+      "api_lib_data_models"
+    ],
     "asyncio": [
       "asyncio_index",
       "asyncio_101",