Skip to content

25. Application Exercise: Version 8

25.1. Introduction

We are going to write a new client/server application. The new feature of the server is that it will manage a session. Instead of placing the tax administration data in an [application] scope object, we will place it in a [session] scope object. Doing so will degrade the code’s performance. When an object can be shared in read-only mode by all users, it is preferable to make it an [application] scope object rather than a [session] scope object. We gain at least some bandwidth since this reduces the size of the session cookie. But we want to demonstrate a client/server application where the client and server exchange a session cookie.

The application architecture remains unchanged:

Image

25.2. The web server

The directory structure of the server scripts is as follows:

Image

The [http-servers/03] folder is initially created by copying the [http-servers/02] folder. Modifications are then made.

25.2.1. The configuration

It is the same as in the |previous version| with a few changes in the [config] script:


# absolute dependencies
    absolute_dependencies = [
        # project folders
        # BaseEntity, MyException
        f"{root_dir}/classes/02/entities",
        # TaxDaoInterface, TaxBusinessInterface, TaxUiInterface
        f"{root_dir}/taxes/v04/interfaces",
        # AbstractTaxDao, TaxConsole, TaxBusiness
        f"{root_dir}/taxes/v04/services",
        # TaxDaoWithAdminDataInDatabase
        f"{root_dir}/taxes/v05/services",
        # AdminData, TaxError, TaxPayer
        f"{root_dir}/impots/v04/entities",
        # Constants, Brackets
        f"{root_dir}/taxes/v05/entities",
        # index_controller
        f"{script_dir}/../controllers",
        # scripts [config_database, config_layers]
        script_dir,
        # Logger, SendAdminMail
        f"{root_dir}/impots/http-servers/02/utilities",
    ]
  • line 17: we will rewrite a controller for the [index] function that handles the / URL;
  • line 21: we use the utilities from the |previous version|;

25.2.2. The main script [main]

The new [main] script introduces a few changes to the main [main] script from the previous version:


# The Flask application can start
app = Flask(__name__)
# session secret key
app.secret_key = os.urandom(12).hex()
  • Line 4: We create a secret key for the application. We know this is necessary for session management;

Next, tax data is no longer requested in the [main] code. The following lines are removed:


# Retrieving data from the tax authority
error = False
try:
    # admindata will be a read-only application-scope variable
    config["admindata"] = config["layers"]["dao"].get_admindata()
    # success log
    logger.write("[server] database connection successful\n")
except ImpôtsError as ex:
    # Log the error
    error = True
    # error log
    log = f"The following error occurred: {ex}"
    # console
    print(log)
    # log file
    logger.write(f"{log}\n")
    # email to the administrator
    send_adminmail(config, log)

Additionally, the [index_controller] controller accepts an additional parameter, the Flask session:


from flask import request, Flask, session
….        
        # execute the request via a controller
        result, status_code = index_controller.execute(request, session, config)

25.2.3. The [index_controller] controller

The [index_controller] controller now manages a session:


# import dependencies
import os
import re
import threading

from flask_api import status
from werkzeug.local import LocalProxy

# Parameterized URL: /?married=xx&children=yy&salary=zz
from AdminData import AdminData
from TaxError import TaxError


def execute(request: LocalProxy, session: LocalProxy, config: dict) -> tuple:
    # dependencies
    from TaxPayer import TaxPayer

    # initially no errors
    errors = []
    …

    # any errors?
    if errors:
        # return an error response to the client
        return {"response": {"errors": errors}}, status.HTTP_400_BAD_REQUEST

    # no errors, we can proceed
    # retrieve the configuration associated with the thread
    thread_name = threading.current_thread().name
    logger = config[thread_name]["config"]["logger"]
    # execute the request
    response = None
    try:
        # the simplest case - admindata is already logged in
        if session.get('client_id') is not None:
            # Retrieve session information
            client_id = session.get('client_id')
            admindata = AdminData().fromdict(session.get('admindata'))
            # log
            logger.write(f"[index_controller] client [{client_id}], tax data retrieved during session\n")
        else:
            # Retrieve data from the tax administration
            admindata = config["layers"]["dao"].get_admindata()
            # load admindata into the session
            session['admindata'] = admindata.asdict()
            # assign a number to the client and store that number in the session
            # This will allow us to track them in the server logs
            client_id = os.urandom(12).hex()
            session['client_id'] = client_id
            # log
            logger.write(f"[index_controller] client [{client_id}], tax data retrieved from the DAO layer\n")
        # tax calculation
        taxpayer = TaxPayer().fromdict({'married': married, 'children': children, 'salary': salary})
        config["layers"]["business"].calculate_tax(taxpayer, admindata)
        # return the response to the client
        return {"response": {"result": taxpayer.asdict()}}, status.HTTP_200_OK
    except (BaseException, TaxError) as error:
        # return the response to the client
        return {"response": {"errors": [f"{error}"]}}, status.HTTP_500_INTERNAL_SERVER_ERROR
  • line 14: the controller receives the current session from the web client;
  • lines 35–38: if the client has a session, then it contains two keys:
    • [client_id]: a client ID (line 37);
    • [admindata]: tax administration data in the form of a dictionary (line 38);
  • line 35: we check if the session contains one of the two expected keys;
  • lines 42–51: case where the client’s session has not yet been initialized;
    • line 43: retrieve tax authority data from the [dao] layer;
    • line 45: this data is placed in the session in the form of a dictionary;
    • line 48: a random number is assigned to the client. This number will be different for each client;
    • line 49: this number is stored in the session;
    • line 51: we log the fact that the tax authority data was retrieved by the [dao] layer. Access to the [dao] layer is generally costly. That is why it must be limited. The idea here is to retrieve the tax data from the [dao] layer once, store it in the session, and fetch it from there during subsequent requests from the same client. Note that this is not the best solution. Since the tax data from the administration is the same for all clients, it belongs in an application-scope object;
  • lines 35–40: case where the client’s session was initialized during a previous request;
    • line 37: retrieve the client ID from the session;
    • line 38: we retrieve the administration’s tax data from the session;
    • line 40: we log the fact that the client has obtained the administration’s tax data from the session;

25.3. The web client

Image

25.3.1. The [dao] layer

25.3.1.1. The [ImpôtsDaoWithHttpSession] class

The [dao] layer is implemented by the following [ImpôtsDaoWithHttpSession] class:


# imports

import requests
from flask_api import status
from myutils import decode_flask_session

from AbstractTaxDao import AbstractTaxDao
from AdminData import AdminData
from TaxError import TaxError
from BusinessTaxInterface import BusinessTaxInterface
from TaxPayer import TaxPayer


class TaxDaoWithHttpSession(AbstractTaxDao, BusinessTaxInterface):

    # constructor
    def __init__(self, config: dict):
        # Initialize parent
        AbstractTaxDao.__init__(self, config)
        # store configuration elements
        # general config
        self.__config = config
        # server
        self.__config_server = config["server"]
        # debug mode
        self.__debug = config["debug"]
        # logger
        self.__logger = None
        # cookies
        self.__cookies = None

    # unused method
    def get_admindata(self) -> AdminData:
        pass

    # tax calculation
    def calculate_tax(self, taxpayer: TaxPayer, admindata: AdminData = None):
        # let exceptions propagate
        # get parameters
        params = {"married": taxpayer.married, "children": taxpayer.children, "salary": taxpayer.salary}
        # connection with Basic Authentication?
        if self.__config_server['authBasic']:
            response = requests.get(
                # URL of the server being queried
                self.__config_server['urlServer'],
                # URL parameters
                params=params,
                # Basic authentication
                auth=(
                    self.__config_server["user"]["login"],
                    self.__config_server["user"]["password"]),
                cookies=self.__cookies)

        else:
            # connection without Basic authentication
            response = requests.get(self.__config_server['urlServer'], params=params, cookies=self.__cookies)
        # retrieve cookies from the response if any
        if response.cookies:
            self.__cookies = response.cookies
            # retrieve the session cookie
            session_cookie = response.cookies.get('session')
            # decode it to log it
            if session_cookie:
                # logger
                if not self.__logger:
                    self.__logger = self.__config['logger']
                # log
                self.__logger.write(f"session cookie={decode_flask_session(session_cookie)}\n")

        # debug mode?
        if self.__debug:
            # logger
            if not self.__logger:
                self.__logger = self.__config['logger']
            # logging
            self.__logger.write(f"{response.text}\n")
        # HTTP response status code
        status_code = response.status_code
        # put the JSON response into a dictionary
        result = response.json()
        # error if status code is not 200 OK
        if status_code != status.HTTP_200_OK:
            # we know that errors have been associated with the [errors] key in the response
            raise TaxError(87, result['response']['errors'])
        # We know that the result has been associated with the [result] key in the response
        # we modify the input parameter with this result
        taxpayer.fromdict(result["response"]["result"])
  • line 30: the [dao] layer will manage a dictionary of cookies;
  • line 58: the [response.cookies] property is a dictionary containing the cookies sent by the server in the [Set-Cookie] HTTP headers;
  • line 59: these cookies are stored in the [dao] layer. They will be sent back to the server during subsequent requests from the same client;
  • lines 60–68: although not strictly necessary, we retrieve the session cookie. In the dictionary of cookies sent by the server, the session cookie is associated with the key [session];
  • lines 62–68: we decode the session cookie to log the user in;
  • line 68: we will return later to the [decode_flask_session] function, which decodes the session cookie;
  • lines 52 and 57: With each request from the same client, the cookies sent by the server are returned to it. This is how the Flask session is maintained between the client and the server;

We must now remember that the [dao] layer will be executed simultaneously by multiple threads. We must therefore examine all the properties of the class instance and see if simultaneous access to these properties poses a problem. Here we have added the [self.__cookies] property on line 30. This property is modified on line 59. We therefore have write access to data shared by all threads. However, this access poses a problem: each thread representing a given client has its own session cookie. In fact, it contains a unique client ID (=thread) for each client. If we do nothing, thread T2 can overwrite the cookies of thread T1.

We have already seen a method to handle this problem: we can create different keys for each thread in the [config] file passed as a parameter to the constructor (line 17). For example, we can use the thread name as the key:

  • line 59, we could write: 

config[thread_name][‘cookies’]=cookies
  • on line 52, we could then write: 

cookies = config[thread_name][‘cookies’]

Here, we’ll use a different technique: each thread (=client) will have its own [dao] layer. This way, line 59 is no longer an issue because the cookies used are those of a single client.

To do this, we will create a new class [ImpôtsDaoWithHttpSessionFactory].

25.3.1.2. The Flask session decoding function

The [decode_flask_session] function is defined in the [myutils] script:

Image

We have already studied the |myutils| script. This script is a machine-scope module that the various scripts in this course can import using the statement:

import myutils

In it, we define the [decode_flask_session] function as follows:


def decode_flask_session(cookie: str) -> str:
    # source: https://www.kirsle.net/wizards/flask-session.cgi
    compressed = False
    payload = cookie

    if payload.startswith('.'):
        compressed = True
        payload = payload[1:]

    data = payload.split(".")[0]

    data = base64_decode(data)
    if compressed:
        data = zlib.decompress(data)

    return data.decode("utf-8")
  • line 2: the URL where I found this function;
  • line 1: the [cookie] parameter is the string associated with the [session] key in the dictionary of cookies received by a web client;
  • lines 3–16: I won’t comment on this code, as I don’t fully understand it;

We add a new import to the [__init__.py] file:


from .myutils import set_syspath, json_response, decode_flask_session

The new version of [myutils] is installed among the machine-wide modules using the [pip install .] command in a PyCharm terminal:


(venv) C:\Data\st-2020\dev\python\cours-2020\python3-flask-2020\packages>pip install .
Processing c:\data\st-2020\dev\python\cours-2020\python3-flask-2020\packages
Using legacy setup.py install for myutils, since the 'wheel' package is not installed.
Installing collected packages: myutils
  Attempting to uninstall: myutils
    Found existing installation: myutils 0.1
    Uninstalling myutils-0.1:
      Successfully uninstalled myutils-0.1
    Running setup.py install for myutils ... done
Successfully installed myutils-0.1
  • Line 1: You must be in the [packages] folder to enter this command;

25.3.1.3. The [ImpôtsDaoWithHttpSessionFactory] class

The [ImpôtsDaoWithHttpSessionFactory] class is as follows:


from ImpôtsDaoWithHttpSession import ImpôtsDaoWithHttpSession


class ImpôtsDaoWithHttpSessionFactory:

    def __init__(self, config: dict):
        # store the parameter
        self.__config = config

    def new_instance(self):
        # return an instance of the [dao] layer
        return TaxDaoWithHttpSession(self.__config)
  • The [ImpôtsDaoWithHttpSessionFactory] class allows us to create a new implementation of the [dao] layer using the [new_instance] method in lines 10–12;

25.3.2. Configuration

The [config_layers] script, which configures the web client layers, is modified as follows:


def configure(config: dict) -> dict:
    # Instantiation of the application layers

    # DAO layer
    from ImpôtsDaoWithHttpSessionFactory import ImpôtsDaoWithHttpSessionFactory
    dao_factory = ImpôtsDaoWithHttpSessionFactory(config)

    # return the layer configuration
    return {
        "dao_factory": dao_factory
    }
  • lines 5-6: instead of instantiating a single [DAO] layer as was done previously, we instantiate a ‘factory’ for this layer (factory = object production factory, in this case the [DAO] layer);
  • lines 9-11: we return the layer configuration;

25.3.3. The client’s main script

The [main] script has changed as follows compared to the previous version:


# configure the application

import config
config = config.configure({})

# dependencies
from ImpôtsError import ImpôtsError
import random
import sys
import threading
from Logger import Logger


# executing the [dao] layer in a thread
# taxpayers is a list of taxpayers
def thread_function(thread_dao, logger, taxpayers: list):
    …


# list of client threads
threads = []
logger = None
# code
try:
    …
    l_taxpayers = len(taxpayers)
    while i < len(taxpayers):
        …
        # Each thread must have its own [DAO] layer to properly manage its session cookie
        thread_dao = dao_factory.new_instance()
        # create the thread
        thread = threading.Thread(target=thread_function, args=(thread_dao, logger, thread_taxpayers))
        # add it to the list of threads in the main script
        threads.append(thread)
        # Start the thread—this operation is asynchronous—we do not wait for the thread's result
        thread.start()
    # The main thread waits for all the threads it has started to finish
    …
except BaseException as error:
    # display the error
    print(f"The following error occurred: {error}")
finally:
    # close the logger
    if logger:
        logger.close()
    # we're done
    print("Work completed...")
    # Terminate any threads that might still be running if we stopped due to an error
    sys.exit()
  • lines 29-30: each thread has its own [dao] layer;

25.3.4. Client execution

The web server is launched, the DBMS is launched, the mail server [hMailServer] is launched. Then we launch the web client's [main] script. The execution logs in [data/logs/logs.txt] are then as follows:


2020-07-25 10:21:46.478511, Thread-1: start of thread [Thread-1] with 1 taxpayer(s)
2020-07-25 10:21:46.479111, Thread-1: Start of tax calculation for {"id": 1, "married": "yes", "children": 2, "salary": 55555}
2020-07-25 10:21:46.479111, Thread-2: Start of thread [Thread-2] with 1 taxpayer(s)
2020-07-25 10:21:46.480195, Thread-3: Start of thread [Thread-3] with 2 taxpayer(s)
2020-07-25 10:21:46.480195, Thread-2: Start of tax calculation for {"id": 2, "married": "yes", "children": 2, "salary": 50000}
2020-07-25 10:21:46.481137, Thread-4: Start of thread [Thread-4] with 3 taxpayers
2020-07-25 10:21:46.481137, Thread-3: Start of tax calculation for {"id": 3, "married": "yes", "children": 3, "salary": 50000}
2020-07-25 10:21:46.482279, Thread-5: Start of thread [Thread-5] with 3 taxpayers
2020-07-25 10:21:46.482622, Thread-6: Start of thread [Thread-6] with 1 taxpayer
2020-07-25 10:21:46.482622, Thread-4: Start of tax calculation for {"id": 5, "married": "no", "children": 3, "salary": 100000}
2020-07-25 10:21:46.485923, Thread-5: Start of tax calculation for {"id": 8, "married": "no", "children": 0, "salary": 100000}
2020-07-25 10:21:46.485923, Thread-6: Start of tax calculation for {"id": 11, "married": "yes", "children": 3, "salary": 200000}
2020-07-25 10:21:46.725910, Thread-4: session cookie={"admindata":{"max_10_percent_allowance":12502.0,"min_10_percent_allowance":437.0,"coeffn":[0.0,1394.96,5798.0,13913.7,20163.4],"coeffr":[0.0,0.14,0.3,0.41,0.45],"id":1,"limits":[9964.0,27519.0,73779.0,156244.0,90000.0],"single_tax_credit_threshold":1196.0,"couple_tax_credit_threshold":1970.0,"single_tax_threshold_for_tax_credit":1595.0,"couple_tax_threshold_for_tax_credit":2627.0,"half-share_income_limit":1551.0,"single_income_limit_for_reduction":21037.0,"couple_income_limit_for_reduction":42074.0,"half-share_reduction_value":3797.0},"client_id":"fa3c83b82761c83e13217967"}
2020-07-25 10:21:46.725910, Thread-4 : {"response": {"result": {"married": "no", "children": 3, "salary": 100000, "tax": 16782, "surcharge": 7176, "rate": 0.41, "discount": 0, "reduction": 0}}}
2020-07-25 10:21:46.725910, Thread-4: End of tax calculation for {"id": 5, "married": "no", "children": 3, "salary": 100000, "tax": 16782, "surcharge": 7176, "rate": 0.41, "discount": 0, "reduction": 0}
2020-07-25 10:21:46.726960, Thread-4: Starting tax calculation for {"id": 6, "married": "yes", "children": 3, "salary": 100000}
2020-07-25 10:21:47.514108, Thread-3: session cookie={"admindata":{"max_10_percent_allowance":12502.0,"min_10_percent_allowance":437.0,"coeffn":[0.0,1394.96,5798.0,13913.7,20163.4],"coeffr":[0.0,0.14,0.3,0.41,0.45],"id":1,"limits":[9964.0,27519.0,73779.0,156244.0,24999.5],"single_tax_credit_threshold":1196.0,"couple_tax_credit_threshold":1970.0,"single_tax_threshold_for_tax_credit":1595.0,"couple_tax_threshold_for_tax_credit":2627.0,"half-share_income_limit":1551.0,"single_income_limit_for_reduction":21037.0,"couple_income_limit_for_reduction":42074.0,"half-share_reduction_value":3797.0},"client_id":"700e3f5dc808c7c48f0c9007"}
2020-07-25 10:21:47.514610, Thread-3 : {"response": {"result": {"married": "yes", "children": 3, "salary": 50000, "tax": 0, "surcharge": 0, "rate": 0.14, "discount": 720, "reduction": 0}}}
2020-07-25 10:21:47.514939, Thread-3: End of tax calculation for {"id": 3, "married": "yes", "children": 3, "salary": 50000, "tax": 0, "surcharge": 0, "rate": 0.14, "discount": 720, "reduction": 0}
2020-07-25 10:21:47.514939, Thread-3: Start of tax calculation for {"id": 4, "married": "no", "children": 2, "salary": 100000}
2020-07-25 10:21:47.527211, Thread-5: session cookie={"admindata":{"max_10_percent_allowance":12502.0,"min_10_percent_allowance":437.0,"coeffn":[0.0,1394.96,5798.0,13913.7,20163.4],"coeffr":[0.0,0.14,0.3,0.41,0.45],"id":1,"limits":[9964.0,27519.0,73779.0,156244.0,90000.0],"single_tax_credit_threshold":1196.0,"couple_tax_credit_threshold":1970.0,"single_tax_threshold_for_tax_credit":1595.0,"couple_tax_threshold_for_tax_credit":2627.0,"half-share_income_limit":1551.0,"single_income_limit_for_reduction":21037.0,"couple_income_limit_for_reduction":42074.0,"half-share_reduction_value":3797.0},"client_id":"9e14a5d4a3057f69ab95ab2d"}
2020-07-25 10:21:47.527211, Thread-2: session cookie={"admindata":{"max_10_percent_deduction":12502.0,"min_10_percent_deduction":437.0,"coeffn":[0.0,1394.96,5798.0,13913.7,20163.4],"coeffr":[0.0,0.14,0.3,0.41,0.45],"id":1,"limits":[9964.0,27519.0,73779.0,156244.0,22500.0],"single_tax_credit_threshold":1196.0,"couple_tax_credit_threshold":1970.0,"single_tax_threshold_for_tax_credit":1595.0,"couple_tax_threshold_for_tax_credit":2627.0,"half-share_income_limit":1551.0,"single_income_limit_for_reduction":21037.0,"couple_income_limit_for_reduction":42074.0,"half-share_reduction_value":3797.0},"client_id":"a06e8fd70a44c9e311f4dce0"}
2020-07-25 10:21:47.527211, Thread-5 : {"response": {"result": {"married": "no", "children": 0, "salary": 100000, "tax": 22986, "surcharge": 0, "rate": 0.41, "discount": 0, "reduction": 0}}}
2020-07-25 10:21:47.527211, Thread-1: session cookie={"admindata":{"max_10_percent_deduction":12502.0,"min_10_percent_deduction":437.0,"coeffn":[0.0,1394.96,5798.0,13913.7,20163.4],"coeffr":[0.0,0.14,0.3,0.41,0.45],"id":1,"limits":[9964.0,27519.0,73779.0,156244.0,90000.0],"single_tax_credit_threshold":1196.0,"couple_tax_credit_threshold":1970.0,"single_tax_threshold_for_tax_credit":1595.0,"couple_tax_threshold_for_tax_credit":2627.0,"half-share_income_limit":1551.0,"single_income_limit_for_reduction":21037.0,"couple_income_limit_for_reduction":42074.0,"half-share_reduction_value":3797.0},"client_id":"28c38df998f67685b3a482b8"}
2020-07-25 10:21:47.527211, Thread-2 : {"response": {"result": {"married": "yes", "children": 2, "salary": 50000, "tax": 1384, "surcharge": 0, "rate": 0.14, "discount": 384, "reduction": 347}}}
2020-07-25 10:21:47.528341, Thread-5: end of tax calculation for {"id": 8, "married": "no", "children": 0, "salary": 100000, "tax": 22986, "surcharge": 0, "rate": 0.41, "discount": 0, "reduction": 0}
2020-07-25 10:21:47.528341, Thread-1: {"response": {"result": {"married": "yes", "children": 2, "salary": 55555, "tax": 2814, "surcharge": 0, "rate": 0.14, "discount": 0, "reduction": 0}}}
2020-07-25 10:21:47.528842, Thread-2: end of tax calculation for {"id": 2, "married": "yes", "children": 2, "salary": 50000, "tax": 1384, "surcharge": 0, "rate": 0.14, "discount": 384, "reduction": 347}
2020-07-25 10:21:47.529349, Thread-5: Start of tax calculation for {"id": 9, "married": "yes", "children": 2, "salary": 30000}
2020-07-25 10:21:47.529699, Thread-1: end of tax calculation for {"id": 1, "married": "yes", "children": 2, "salary": 55555, "tax": 2814, "surcharge": 0, "rate": 0.14, "discount": 0, "reduction": 0}
2020-07-25 10:21:47.529699, Thread-2: end of thread [Thread-2]
2020-07-25 10:21:47.531905, Thread-1: end of thread [Thread-1]
2020-07-25 10:21:47.536121, Thread-6: session cookie={"admindata":{"max_10_percent_discount":12502.0,"min_10_percent_discount":437.0,"coeffn":[0.0,1394.96,5798.0,13913.7,20163.4],"coeffr":[0.0,0.14,0.3,0.41,0.45],"id":1,"limits":[9964.0,27519.0,73779.0,156244.0,93749.0],"single_tax_credit_threshold":1196.0,"couple_tax_credit_threshold":1970.0,"single_tax_threshold_for_tax_credit":1595.0,"couple_tax_threshold_for_tax_credit":2627.0,"half-share_income_limit":1551.0,"single_income_limit_for_reduction":21037.0,"couple_income_limit_for_reduction":42074.0,"half-share_reduction_value":3797.0},"client_id":"38499b63076516c02f2770ec"}
2020-07-25 10:21:47.537161, Thread-3 : {"response": {"result": {"married": "no", "children": 2, "salary": 100000, "tax": 19884, "surcharge": 4480, "rate": 0.41, "discount": 0, "reduction": 0}}}
2020-07-25 10:21:47.537161, Thread-6 : {"response": {"result": {"married": "yes", "children": 3, "salary": 200000, "tax": 42842, "surcharge": 17283, "rate": 0.41, "discount": 0, "reduction": 0}}}
2020-07-25 10:21:47.538156, Thread-3: End of tax calculation for {"id": 4, "married": "no", "children": 2, "salary": 100000, "tax": 19884, "surcharge": 4480, "rate": 0.41, "discount": 0, "reduction": 0}
2020-07-25 10:21:47.538557, Thread-6: End of tax calculation for {"id": 11, "married": "yes", "children": 3, "salary": 200000, "tax": 42842, "surcharge": 17283, "rate": 0.41, "discount": 0, "reduction": 0}
2020-07-25 10:21:47.538828, Thread-3: end of thread [Thread-3]
2020-07-25 10:21:47.538828, Thread-6: end of thread [Thread-6]
2020-07-25 10:21:47.546198, Thread-5: {"response": {"result": {"married": "yes", "children": 2, "salary": 30000, "tax": 0, "surcharge": 0, "rate": 0.0, "discount": 0, "reduction": 0}}}
2020-07-25 10:21:47.546198, Thread-5: end of tax calculation for {"id": 9, "married": "yes", "children": 2, "salary": 30000, "tax": 0, "surcharge": 0, "rate": 0.0, "discount": 0, "reduction": 0}
2020-07-25 10:21:47.546198, Thread-5: Start of tax calculation for {"id": 10, "married": "no", "children": 0, "salary": 200000}
2020-07-25 10:21:47.739643, Thread-4: {"response": {"result": {"married": "yes", "children": 3, "salary": 100000, "tax": 9200, "surcharge": 2180, "rate": 0.3, "discount": 0, "reduction": 0}}}
2020-07-25 10:21:47.739643, Thread-4: End of tax calculation for {"id": 6, "married": "yes", "children": 3, "salary": 100000, "tax": 9200, "surcharge": 2180, "rate": 0.3, "discount": 0, "reduction": 0}
2020-07-25 10:21:47.740668, Thread-4: Starting tax calculation for {"id": 7, "married": "yes", "children": 5, "salary": 100000}
2020-07-25 10:21:48.557469, Thread-5: {"response": {"result": {"married": "no", "children": 0, "salary": 200000, "tax": 64210, "surcharge": 7498, "rate": 0.45, "discount": 0, "reduction": 0}}}
2020-07-25 10:21:48.558715, Thread-5: end of tax calculation for {"id": 10, "married": "no", "children": 0, "salary": 200000, "tax": 64210, "surcharge": 7498, "rate": 0.45, "discount": 0, "reduction": 0}
2020-07-25 10:21:48.558715, Thread-5: end of thread [Thread-5]
2020-07-25 10:21:48.753025, Thread-4 : {"response": {"result": {"married": "yes", "children": 5, "salary": 100000, "tax": 4230, "surcharge": 0, "rate": 0.14, "discount": 0, "reduction": 0}}}
2020-07-25 10:21:48.753318, Thread-4: End of tax calculation for {"id": 7, "married": "yes", "children": 5, "salary": 100000, "tax": 4230, "surcharge": 0, "rate": 0.14, "discount": 0, "reduction": 0}
2020-07-25 10:21:48.753540, Thread-4: end of thread [Thread-4]
  • There are a total of 6 threads, meaning 6 clients (lines 1, 3, 4, 6, 8, 9) that simultaneously query the tax calculation server;
  • we will follow thread [Thread-4], which handles 3 taxpayers (line 6). It will make three sequential requests to the tax calculation server;
  • Line 10: [Thread-4]’s first request;
  • line 13: [Thread-4] has received the response to its first request. Inside, it finds a session cookie containing the number [fa3c83b82761c83e13217967] assigned to it by the server;
  • line 14: the tax for the first taxpayer;
  • line 16: [Thread-4] makes a request for the second taxpayer;
  • line 43: [Thread-4] receives the tax amount for the second taxpayer;
  • line 45: [Thread-4] makes a request for the third taxpayer;
  • line 49: [Thread-4] receives the tax amount for the third taxpayer;
  • line 51: [Thread-4] has finished its work;

Now, let’s look at how the three requests from [Thread-4] were processed on the server side. We can track them in the server logs using its client ID [fa3c83b82761c83e13217967].

The server-side logs [data/logs/logs.txt] are as follows:


2020-07-25 10:21:39.187366, MainThread: [server] server startup
2020-07-25 10:21:40.439093, MainThread: [server] server startup
2020-07-25 10:21:46.502011, Thread-2: [index] request: <Request 'http://127.0.0.1:5000/?marié=oui&enfants=3&salaire=50000' [GET]>
2020-07-25 10:21:46.504049, Thread-2: [index] thread paused for 1 second(s)
2020-07-25 10:21:46.505452, Thread-3: [index] request: <Request 'http://127.0.0.1:5000/?marié=oui&enfants=2&salaire=55555' [GET]>
2020-07-25 10:21:46.506257, Thread-3: [index] Thread paused for 1 second
2020-07-25 10:21:46.507292, Thread-4: [index] request: <Request 'http://127.0.0.1:5000/?marié=non&enfants=0&salaire=100000' [GET]>
2020-07-25 10:21:46.507292, Thread-4: [index] Thread paused for 1 second
2020-07-25 10:21:46.508301, Thread-5: [index] request: <Request 'http://127.0.0.1:5000/?marié=oui&enfants=2&salaire=50000' [GET]>
2020-07-25 10:21:46.509293, Thread-5: [index] Thread paused for 1 second
2020-07-25 10:21:46.511808, Thread-6: [index] request: <Request 'http://127.0.0.1:5000/?marié=non&enfants=3&salaire=100000' [GET]>
2020-07-25 10:21:46.517604, Thread-7: [index] request: <Request 'http://127.0.0.1:5000/?marié=oui&enfants=3&salaire=200000' [GET]>
2020-07-25 10:21:46.517604, Thread-7: [index] Thread paused for 1 second
2020-07-25 10:21:46.719504, Thread-6: [index_controller] client [fa3c83b82761c83e13217967], tax data retrieved from the DAO layer
2020-07-25 10:21:46.720003, Thread-6 : [index] {'response': {'result': {'married': 'no', 'children': 3, 'salary': 100000, 'tax': 16782, 'surcharge': 7176, 'rate': 0.41, 'discount': 0, 'reduction': 0}}}
2020-07-25 10:21:46.736108, Thread-8: [index] request: <Request 'http://127.0.0.1:5000/?marié=oui&enfants=3&salaire=100000' [GET]>
2020-07-25 10:21:46.736108, Thread-8 : [index] paused thread for 1 second(s)
2020-07-25 10:21:47.506709, Thread-2: [index_controller] client [700e3f5dc808c7c48f0c9007], tax data retrieved from the DAO layer
2020-07-25 10:21:47.507216, Thread-2: [index] {'response': {'result': {'married': 'yes', 'children': 3, 'salary': 50000, 'tax': 0, 'surcharge': 0, 'rate': 0.14, 'discount': 720, 'reduction': 0}}}
2020-07-25 10:21:47.507216, Thread-3: [index_controller] client [28c38df998f67685b3a482b8], tax data retrieved from the DAO layer
2020-07-25 10:21:47.508442, Thread-4: [index_controller] client [9e14a5d4a3057f69ab95ab2d], tax data retrieved from the DAO layer
2020-07-25 10:21:47.508940, Thread-3: [index] {'response': {'result': {'married': 'yes', 'children': 2, 'salary': 55555, 'tax': 2814, 'surcharge': 0, 'rate': 0.14, 'discount': 0, 'reduction': 0}}}
2020-07-25 10:21:47.510506, Thread-4: [index] {'response': {'result': {'married': 'no', 'children': 0, 'salary': 100000, 'tax': 22986, 'surcharge': 0, 'rate': 0.41, 'discount': 0, 'reduction': 0}}}
2020-07-25 10:21:47.511513, Thread-5: [index_controller] client [a06e8fd70a44c9e311f4dce0], tax data retrieved from the DAO layer
2020-07-25 10:21:47.514939, Thread-5 : [index] {'response': {'result': {'married': 'yes', 'children': 2, 'salary': 50000, 'tax': 1384, 'surcharge': 0, 'rate': 0.14, 'discount': 384, 'reduction': 347}}}
2020-07-25 10:21:47.520727, Thread-7: [index_controller] client [38499b63076516c02f2770ec], tax data retrieved from the DAO layer
2020-07-25 10:21:47.523162, Thread-7 : [index] {'response': {'result': {'married': 'yes', 'children': 3, 'salary': 200000, 'tax': 42842, 'surcharge': 17283, 'rate': 0.41, 'discount': 0, 'reduction': 0}}}
2020-07-25 10:21:47.530835, Thread-9: [index] request: <Request 'http://127.0.0.1:5000/?marié=non&enfants=2&salaire=100000' [GET]>
2020-07-25 10:21:47.531736, Thread-9 : [index_controller] client [700e3f5dc808c7c48f0c9007], tax data retrieved during session
2020-07-25 10:21:47.531905, Thread-9 : [index] {'response': {'result': {'married': 'no', 'children': 2, 'salary': 100000, 'tax': 19884, 'surcharge': 4480, 'rate': 0.41, 'discount': 0, 'reduction': 0}}}
2020-07-25 10:21:47.541899, Thread-10: [index] request: <Request 'http://127.0.0.1:5000/?marié=oui&enfants=2&salaire=30000' [GET]>
2020-07-25 10:21:47.542488, Thread-10 : [index_controller] client [9e14a5d4a3057f69ab95ab2d], tax data retrieved during session
2020-07-25 10:21:47.542488, Thread-10 : [index] {'response': {'result': {'married': 'yes', 'children': 2, 'salary': 30000, 'tax': 0, 'surcharge': 0, 'rate': 0.0, 'discount': 0, 'reduction': 0}}}
2020-07-25 10:21:47.553628, Thread-11: [index] request: <Request 'http://127.0.0.1:5000/?marié=non&enfants=0&salaire=200000' [GET]>
2020-07-25 10:21:47.553628, Thread-11 : [index] thread paused for 1 second(s)
2020-07-25 10:21:47.736910, Thread-8: [index_controller] client [fa3c83b82761c83e13217967], tax data retrieved during session
2020-07-25 10:21:47.737191, Thread-8: [index] {'response': {'result': {'married': 'yes', 'children': 3, 'salary': 100000, 'tax': 9200, 'surcharge': 2180, 'rate': 0.3, 'discount': 0, 'reduction': 0}}}
2020-07-25 10:21:47.748226, Thread-12: [index] request: <Request 'http://127.0.0.1:5000/?marié=oui&enfants=5&salaire=100000' [GET]>
2020-07-25 10:21:47.748226, Thread-12 : [index] thread paused for 1 second(s)
2020-07-25 10:21:48.554695, Thread-11: [index_controller] client [9e14a5d4a3057f69ab95ab2d], tax data retrieved during session
2020-07-25 10:21:48.555070, Thread-11: [index] {'response': {'result': {'married': 'no', 'children': 0, 'salary': 200000, 'tax': 64210, 'surcharge': 7498, 'rate': 0.45, 'discount': 0, 'reduction': 0}}}
2020-07-25 10:21:48.748753, Thread-12: [index_controller] client [fa3c83b82761c83e13217967], tax data retrieved during session
2020-07-25 10:21:48.748753, Thread-12 : [index] {'response': {'result': {'married': 'yes', 'children': 5, 'salary': 100000, 'tax': 4230, 'surcharge': 0, 'rate': 0.14, 'discount': 0, 'reduction': 0}}}
  • The client [fa3c83b82761c83e13217967] is encountered for the first time on line 14: to calculate the tax, the server had to retrieve data from the tax authority’s database;
  • then we see the client [fa3c83b82761c83e13217967] again on line 36. This time, the server retrieves the tax authority data from the session, which saves it a potentially costly access to the [DAO] layer;
  • We encounter the client [fa3c83b82761c83e13217967] a third time on line 42, where once again the server uses the client’s session;

This example clearly illustrates the value of the session for a client: it stores data shared by all of that client’s requests, which is costly to retrieve.

On the client side, the results in the file [data/output/results.json] are the same as in previous versions.

25.4. Testing the [DAO] layer

As we did in the |previous versions|, we test the client’s [dao] layer:

Image

The test class will be executed in the following environment:

Image

  • Configuration [2] is identical to configuration [1], which we just examined;

The [TestHttpClientDao] test class is as follows:


import unittest

from Logger import Logger


class TestHttpClientDao(unittest.TestCase):

    def test_1(self) -> None:
        from TaxPayer import TaxPayer

        # {'married': 'yes', 'children': 2, 'salary': 55555,
        # 'tax': 2814, 'surcharge': 0, 'discount': 0, 'reduction': 0, 'rate': 0.14}
        taxpayer = TaxPayer().fromdict({"married": "yes", "children": 2, "salary": 55555})
        dao.calculate_tax(taxpayer)
        # verification
        self.assertAlmostEqual(taxpayer.tax, 2815, delta=1)
        self.assertEqual(taxpayer.discount, 0)
        self.assertEqual(taxpayer.reduction, 0)
        self.assertAlmostEqual(taxpayer.rate, 0.14, delta=0.01)
        self.assertEqual(taxpayer.surcharge, 0)
…


if __name__ == '__main__':
    # configure the application
    import config
    config = config.configure({})

    # logger
    logger = Logger(config["logsFilename"])
    # Store it in the config
    config["logger"] = logger
    # retrieve the [DAO] layer factory
    dao_factory = config["layers"]["dao_factory"]
    # create an instance of the [dao] layer
    dao = dao_factory.new_instance()

    # run the test methods
    print("Tests in progress...")
    unittest.main()
  • We create an |execution configuration| for this test;
  • We start the web server with its entire environment;
  • run the test;

The results are as follows:


C:\Data\st-2020\dev\python\cours-2020\python3-flask-2020\venv\Scripts\python.exe C:/Data/st-2020/dev/python/cours-2020/python3-flask-2020/impots/http-clients/03/tests/TestHttpClientDao.py
tests in progress...
...........
----------------------------------------------------------------------
Ran 11 tests in 3.392s

OK

Process finished with exit code 0
https://tahe.developpez.com
This tutorial written by Serge Tahé is made available to the public under the terms of the Creative Commons Attribution – Non-Commercial – ShareAlike 3.0 Unported.