5. Resolving the three problems with ChatGPT

5.1. Introduction

Here is a first screenshot of a ChatGPT session:

In [1-3], the three problems posed to ChatGPT;
In [4], the URL from ChatGPT;
In [5], the version from ChatGPT is used;

ChatGPT is a product of OpenAI available at URL [https://chatgpt.com/]. To view a history of your Q&A sessions as shown above, you need to create an account. Furthermore, like all other IA versions tested, ChatGPT limits the number of questions you can ask and the number of files you can upload. When this limit is reached, the session ends and you are prompted to continue it at a later time. The limits imposed by ChatGPT are reached very quickly. To create this tutorial, I had to purchase a one-month paid subscription.

The ChatGPT interface is as follows:

In [1], to attach files to the question asked;
In [2], the question asked;
In [3], to run IA;

5.2. Problem 1

The question in ChatGPT:

ChatGPT answers correctly.

5.3. Problem 2

This involves calculating the tax using PDF. To be honest, we’ll use the PDF generated by Gemini, which corrects errors in the initial PDF.

In [1], we provided the PDF generated by Gemini;
In [2], we added the unit test through which Gemini demonstrated its superiority:

1	`test12 : (2, 2, 49500) -> (1297, 431, 324)`

We run ChatGPT. It takes about 3 minutes to generate its response. Unlike Gemini, it does provide a working link to retrieve the generated script. We load this into PyCharm:

The [chatGPT1] script works on the first try. There’s no contest here; on this problem, ChatGPT performed better than Gemini.

The [chatGPT1] script provided by ChatGPT is as follows:

# -*- coding: utf-8 -*-
"""
Calcul de l'impôt 2019 (revenus 2018) en Python.

Hypothèses et règles (d'après le PDF fourni) :
- Tous les revenus sont portés sur le déclarant 1 (même pour un couple).
- Abattement forfaitaire de 10 % encadré par un minimum et un maximum.
- Barème progressif 2018 appliqué au quotient familial (QF).
- Plafonnement du quotient familial : avantage par demi-part enfant limité.
- Décote et réduction de 20 % pour faibles revenus, avec la règle de repêchage.
- Ordre des correctifs : plafonnement QF -> décote -> réduction 20 %.
- Arrondis : abattement (↑), décote (↑), réduction (↑), impôt final (↓).
- On renvoie l'impôt à payer (arrondi à l'euro inf.), la décote et la réduction
  (arrondies à l'euro sup.).

Les nombres “magiques” sont regroupés en constantes, et chaque étape est isolée
dans des fonctions pour la clarté.
"""

from dataclasses import dataclass
from math import ceil, floor

# ---------- Regulatory constants ----------

# Professional expense allowance (floor/ceiling) for 2018 income
ABATTEMENT_TAUX = 0.10
ABATTEMENT_MIN = 437
ABATTEMENT_MAX = 12502

# Scale 2018 (QF)
# Each band is (borne_sup, rate). The first 0% bracket ends at €9,964.
BAREME_2018 = [
    (9964, 0.00),
    (27519, 0.14),
    (73779, 0.30),
    (156244, 0.41),
    (float("inf"), 0.45),
]

# Family allowance ceiling
PF_QF_AVANTAGE_PAR_DEMI_PART = 1551  # max benefit per 1/2 share ENFANT

# Discount (thresholds according to number of adults / couple status)
DECOTE_SEUIL_CELIB = 1196
DECOTE_SEUIL_COUPLE = 1970

# 20% reduction: share-based thresholds
REDUCTION_BASE_1_PART = 18985
REDUCTION_BASE_2_PARTS = 37970
REDUCTION_PAR_DEMI_PART_SUP = 3803  # addition of 1/2 share above the base of 1 or 2 shares


# ---------- Data structures ----------

@dataclass
class ResultatImpot:
    impot: int  # final tax payable, rounded down to the nearest euro
    decote: int  # discount rounded up to the nearest euro
    reduction: int  # 20% discount rounded up to the nearest euro


# ---------- Utility functions ----------

def arrondi_sup(euros: float) -> int:
    """Arrondi à l'euro supérieur (ceil)."""
    return int(ceil(euros))


def arrondi_inf(euros: float) -> int:
    """Arrondi à l'euro inférieur (floor)."""
    return int(floor(euros))


# ---------- Step 1: Income Net Taxable (RNI) ----------

def calcul_abattement(revenus_bruts: float) -> int:
    """
    Calcule l'abattement de 10 %, plafonné/planche, ARRONDI À L'EURO SUPÉRIEUR.
    """
    abattement_calcule = revenus_bruts * ABATTEMENT_TAUX
    abattement_calcule = max(ABATTEMENT_MIN, min(ABATTEMENT_MAX, abattement_calcule))
    return arrondi_sup(abattement_calcule)


def revenu_net_imposable(revenus_bruts: float) -> int:
    """RNI = revenus bruts - abattement (abattement arrondi ↑)."""
    return int(round(revenus_bruts - calcul_abattement(revenus_bruts)))


# ---------- Step 2: Number of units ----------

def nombre_parts(adultes: int, enfants: int) -> float:
    """
    Règles : 1 part si 1 adulte, 2 parts si 2 adultes.
    Enfants : +0,5 pour les 2 premiers, +1 par enfant à partir du 3e.
    (On ignore ici tout cas particulier type parent isolé, conformément au PDF.)
    """
    parts = 1.0 if adultes == 1 else 2.0
    if enfants <= 0:
        return parts
    if enfants == 1:
        return parts + 0.5
     # children >= 2
    parts += 1.0  # +0.5 +0.5 for the first two
    if enfants > 2:
        parts += (enfants - 2) * 1.0  # +1 per child from the 3rd
    return parts


def demi_parts_enfants(enfants: int) -> int:
    """Nombre de DEMI-parts dû aux enfants (utile pour le plafonnement QF)."""
    if enfants <= 0:
        return 0
    if enfants == 1:
        return 1  # 0.5 part
     # First two children = 2 * half-parts, then +2 half-parts per child from the 3rd onwards
    return 2 + (enfants - 2) * 2


# ---------- Step 3: Gross tax via QF ----------

def impot_brut_par_parts(qf: float) -> float:
    """Impôt par part via barème progressif 2018 (sans arrondi)."""
    imp = 0.0
    previous = 0.0
    for borne_sup, taux in BAREME_2018:
        montant_tranche = max(0.0, min(qf, borne_sup) - previous)
        imp += montant_tranche * taux
        if qf <= borne_sup:
            break
        previous = borne_sup
    return imp


def impot_brut_total(rni: float, parts: float) -> float:
    """Impôt brut total = impôt par part * nombre de parts (sans arrondi)."""
    if parts <= 0:
        return 0.0
    qf = rni / parts
    return impot_brut_par_parts(qf) * parts


# ---------- Step 4.1: Capping the family quotient ----------

def apply_plafonnement_qf(impot_avec_enfants: float, rni: float, adultes: int, enfants: int) -> float:
    """
    Calcule l'impôt après plafonnement QF si l'avantage des enfants dépasse le plafond.
    - Impôt A : avec parts incluant enfants
    - Impôt B : avec seulement parts des adultes (1 ou 2)
    - Avantage réel = B - A
    - Avantage maximal = 1551 € par 1/2 part ENFANT
    Si avantage réel > avantage max, impôt = B - avantage max, sinon impôt = A.
    """
    parts_adultes = 1.0 if adultes == 1 else 2.0
    imp_b = impot_brut_total(rni, parts_adultes)
    imp_a = impot_avec_enfants
    avantage_reel = max(0.0, imp_b - imp_a)
    avantage_max = PF_QF_AVANTAGE_PAR_DEMI_PART * demi_parts_enfants(enfants)
    if avantage_reel > avantage_max:
        return imp_b - avantage_max
    return imp_a


# ---------- Step 4.2: Discount and 20% reduction ----------

def seuil_decote(adultes: int) -> int:
    return DECOTE_SEUIL_COUPLE if adultes >= 2 else DECOTE_SEUIL_CELIB


def seuil_reduction_20(rni: int, adultes: int, parts: float) -> int:
    """
    Seuil d'éligibilité à la réduction de 20 % :
    - Base : 18 985 € pour 1 part, 37 970 € pour 2 parts
    - + 3 803 € par 1/2 part supplémentaire au-delà de la base correspondante.
    """
    base = REDUCTION_BASE_2_PARTS if adultes >= 2 else REDUCTION_BASE_1_PART
    base_parts = 2.0 if adultes >= 2 else 1.0
    demi_sup = max(0, int(round((parts - base_parts) * 2)))
    return base + demi_sup * REDUCTION_PAR_DEMI_PART_SUP


def calc_decote_reduction(impot_apres_plaf: float, rni: int, adultes: int, parts: float):
    """
    Calcule (decote_arrondie, reduction_arrondie, impot_apres_correctifs).
    Règle d'interaction : la décote peut s'appliquer via repêchage si, après
    application THÉORIQUE de la réduction de 20 %, l'impôt passerait sous le seuil.
    Ordre : décote -> réduction.
    """
    imp = max(0.0, impot_apres_plaf)
    decote = 0
    reduction = 0

     # Eligible for 20% discount based on RNI
    seuil_red = seuil_reduction_20(rni, adultes, parts)
    eligible_reduction = rni < seuil_red  # "below a threshold" in PDF

     # Eligible for discount: directly below threshold, or if reduction would cause the threshold to be lowered
    s_dec = seuil_decote(adultes)
    direct_decote = imp <= s_dec
    repechage = False
    if not direct_decote and eligible_reduction:
        imp_theorique_apres_red = imp * 0.80  # theoretical 20% reduction
        if imp_theorique_apres_red <= s_dec:
            repechage = True

    elig_decote = (direct_decote or repechage) and (imp > 0)

     # Apply discount first (if eligible)
    if elig_decote:
        montant_decote = s_dec - (imp * 0.75)
        decote_calc = max(0, arrondi_sup(montant_decote))
         # The discount cannot exceed the remaining tax: it is limited to
        decote = min(decote_calc, arrondi_sup(imp))
        imp = max(0.0, imp - decote)

     # Then apply the reduction if eligible
    if eligible_reduction and imp > 0:
        montant_reduction = imp * 0.20
        reduction = max(0, arrondi_sup(montant_reduction))
        imp = max(0.0, imp - reduction)

    return decote, reduction, imp


# ---------- API main ----------

def calcule_impot(adultes: int, enfants: int, revenus_bruts: float) -> ResultatImpot:
    """
    Calcule l'impôt 2019 (revenus 2018) pour un foyer.
    Retourne ResultatImpot(impot, decote, reduction).
    """
     # 1) RNI
    rni = revenu_net_imposable(revenus_bruts)

     # 2) Units
    parts = nombre_parts(adultes, enfants)

     # 3) Gross tax via tax scale (with all shares)
    imp_brut_A = impot_brut_total(rni, parts)

     # 4.1) Family allowance ceiling
    imp_apres_plaf = apply_plafonnement_qf(imp_brut_A, rni, adultes, enfants)

     # 4.2) Discount then 20% reduction (with draft)
    decote, reduction, imp_corrige = calc_decote_reduction(imp_apres_plaf, rni, adultes, parts)

     # 5) Final tax rounding (↓) and bounding to 0
    imp_final = max(0, arrondi_inf(imp_corrige))

    return ResultatImpot(impot=imp_final, decote=decote, reduction=reduction)


# ---------- Small test executable ----------

def _nearly_equal(a: int, b: int, tol: int = 1) -> bool:
    return abs(a - b) <= tol


def tests_unitaires():
    """
    Renvoie une liste de tuples (inputs, attendu, obtenu, ok) pour chaque test.
    Tolérance : ±1 € sur chaque valeur (impôt, décote, réduction).
    """
    cas = [
         # (adults, children, income) -> (tax, discount, reduction)
        ((2, 2, 55555), (2815, 0, 0)),
        ((2, 2, 50000), (1385, 384, 346)),
        ((2, 3, 50000), (0, 720, 0)),
        ((1, 2, 100000), (19884, 0, 0)),
        ((1, 3, 100000), (16782, 0, 0)),
        ((2, 3, 100000), (9200, 0, 0)),
        ((2, 5, 100000), (4230, 0, 0)),
        ((1, 0, 100000), (22986, 0, 0)),
        ((2, 2, 30000), (0, 0, 0)),
        ((1, 0, 200000), (64211, 0, 0)),
        ((2, 3, 200000), (42843, 0, 0)),
        ((2, 2, 49500), (1297, 431, 324)),
    ]

    resultats = []
    for (adultes, enfants, revenus), attendu in cas:
        res = calcule_impot(adultes, enfants, revenus)
        obtenu = (res.impot, res.decote, res.reduction)
        ok = _nearly_equal(obtenu[0], attendu[0]) and _nearly_equal(obtenu[1], attendu[1]) and _nearly_equal(obtenu[2],
                                                                                                             attendu[2])
        resultats.append(((adultes, enfants, revenus), attendu, obtenu, ok))
    return resultats


if __name__ == "__main__":
    for inputs, attendu, obtenu, ok in tests_unitaires():
        print(f"{inputs} -> attendu={attendu}, obtenu={obtenu} : {'OK' if ok else 'ECHEC'}")

5.4. Problem 3

Now we ask ChatGPT to search for tax calculation rules on the internet:

This time, we do not provide the PDF file that contained the calculation rules to follow. We only provide our instructions in the text file. Note that this text file now contains 12 unit tests after adding, to the initial 11 tests, the one used by Gemini to demonstrate that my initial PDF was incorrect.

ChatGPT responds in 8 minutes, providing a link to download the generated script. Once loaded into PyCharm, this script passes all 12 tests. Thus, for the two problems posed, ChatGPT answered correctly on the first try, thereby outperforming Gemini.

ChatGPT provides its sources in its response:

There’s nothing more to say—it’s a job well done.

Now, we can ask it, just as we did with Gemini, to generate a PDF for students.

The response from ChatGPT was obtained after several back-and-forths because the generated PDF used a font that replaced characters with a square. But eventually, it generated the PDF. I’m including it because it produces different rules than Gemini’s PDF, and I wondered which one was correct. We’ll look into it.

The difference with Gemini’s PDF lies in the calculation of the discount. The two IA files do not take the same approach. Gemini had written:

The two IAs have two different approaches. Who is right?

5.5. Problem 4

We’ll ask ChatGPT to use its PDF to calculate the tax:

As in previous instances, it generates a Python script that works on the first try. We had added an additional test to the instructions:

1	`test13 : (1, 0, 18535) -> (359, 491, 90)`

All 13 tests were passed successfully.

5.6. Back to Gemini

Now, we return to Gemini, to which we will present PDF from ChatGPT. Since the rules implemented in this PDF differ from those implemented in Gemini’s PDF, one might wonder what will happen:

Gemini first generated a Python script that failed tests. We showed it the logs:

Question 2

Question 3

There are still errors. Let’s continue.

Question 4

Still running into errors:

This time it’s correct.

We’re still puzzled that, even though PDF and IA have quite different calculation rules, both generate correct results.

We ask Gemini the following question:

The full question is as follows:

Après tout ce qu'on a fait, compare les deux approches, la tienne et celle de ChatGPT dans les pDF que je te joins [.. selon Gemini] pour toi, [.. selon ChatGPT] pour ChatGPT. Dis-moi laquelle on doit préférer vu que sur les 13 tests générés tu les réussis tous avec l'un ou avec l'autre PDF.

Gemini’s answer is categorical:

5.7. What does ChatGPT think?

We ask ChatGPT the same question we asked Gemini.

ChatGPT’s response is as follows:

So, ChatGPT suggests a unit test to decide between the two methods. We duplicate:

The script [gemini3] generated by Gemini using its PDF as the source [Le problème selon Gemini] is duplicated in the script [gemini4];
The script [chatGPT3] generated by ChatGPT using PDF and [Le problème selon ChatGPT] as sources is duplicated in the script [chatGPT4];

In addition, the unit test proposed by ChatGPT is added to each of the [gemini4, chatGPT4] scripts to distinguish between the two IA scripts.

Running [gemini4] yields the following results:


C:\Data\st-2025\dev\python\code\python-flask-2025-cours\.venv\Scripts\python.exe "C:/Program Files/JetBrains/PyCharm 2025.2.1.1/plugins/python-ce/helpers/pycharm/_jb_unittest_runner.py" --path "C:\Data\st-2025\dev\python\code\python-flask-2025-cours\outils ia\gemini\gemini4.py" 
Testing started at 17:45 ...
Launching unittests with arguments python -m unittest C:\Data\st-2025\dev\python\code\python-flask-2025-cours\outils ia\gemini\gemini4.py in C:\Data\st-2025\dev\python\code\python-flask-2025-cours
 
SubTest failure: Traceback (most recent call last):
  File "C:\Program Files\Python313\Lib\unittest\case.py", line 58, in testPartExecutor
    yield
  File "C:\Program Files\Python313\Lib\unittest\case.py", line 556, in subTest
    yield
  File "C:\Data\st-2025\dev\python\code\python-flask-2025-cours\outils ia\gemini\gemini4.py", line 234, in test_cas_verifies_simulateur_officiel
    self.assertAlmostEqual(calcul_impot, attendu_impot, delta=1, msg="Échec sur le montant de l'impôt")
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2669 != 2270 within 1 delta (399 difference) : Échec sur le montant de l'impôt
 
 
 
 
Ran 1 test in 0.010s
 
FAILED (failures=1)
 
One or more subtests failed
Failed subtests list: [Test 'test12' avec entrée (2, 0, 43333)]
 
Process finished with exit code 1

So Gemini fails the test added by ChatGPT.

Running [chatGPT4] yields the following results:


C:\Data\st-2025\dev\python\code\python-flask-2025-cours\.venv\Scripts\python.exe "C:\Data\st-2025\dev\python\code\python-flask-2025-cours\outils ia\chatGPT\chatGPT4.py" 
Test (2, 2, 55555) -> obtenu (impôt=2814, décote=0, réduction=0) | attendu (2815, 0, 0) | OK
Test (2, 2, 50000) -> obtenu (impôt=1384, décote=384, réduction=347) | attendu (1385, 384, 346) | OK
Test (2, 3, 50000) -> obtenu (impôt=0, décote=721, réduction=0) | attendu (0, 720, 0) | OK
Test (1, 2, 100000) -> obtenu (impôt=19884, décote=0, réduction=0) | attendu (19884, 0, 0) | OK
Test (1, 3, 100000) -> obtenu (impôt=16782, décote=0, réduction=0) | attendu (16782, 0, 0) | OK
Test (2, 3, 100000) -> obtenu (impôt=9200, décote=0, réduction=0) | attendu (9200, 0, 0) | OK
Test (2, 5, 100000) -> obtenu (impôt=4230, décote=0, réduction=0) | attendu (4230, 0, 0) | OK
Test (1, 0, 100000) -> obtenu (impôt=22986, décote=0, réduction=0) | attendu (22986, 0, 0) | OK
Test (2, 2, 30000) -> obtenu (impôt=0, décote=0, réduction=0) | attendu (0, 0, 0) | OK
Test (1, 0, 200000) -> obtenu (impôt=64210, décote=0, réduction=0) | attendu (64211, 0, 0) | OK
Test (2, 3, 200000) -> obtenu (impôt=42842, décote=0, réduction=0) | attendu (42843, 0, 0) | OK
Test (2, 2, 49500) -> obtenu (impôt=1296, décote=431, réduction=325) | attendu (1297, 431, 324) | OK
Test (1, 0, 18535) -> obtenu (impôt=359, décote=491, réduction=90) | attendu (359, 491, 90) | OK
Test (2, 0, 43333) -> obtenu (impôt=2268, décote=0, réduction=401) | attendu (2270, 0, 400) | ECHEC
 Détails tolérance ±1€ : impôt ok? False, décote ok? True, réduction ok? True
 
Résultat global : AU MOINS UN TEST ÉCHOUE ❌
 
Process finished with exit code 0

ChatGPT also fails the added test but not for the same reasons as Gemini. ChatGPT found the correct results but was off by 2 euros instead of the required 1 euro.

So from now on, we will use the PDF generated by ChatGPT with the subsequent IA files. It should be noted that it is because of the lack of unit tests in my instructions that both IAs passed the initial tests. Hence, in this specific example, the importance of including unit tests for edge cases in tax calculation. Since it’s rather difficult to come up with these tests on your own. We’ll ask the IA to add them themselves.

5.8. Problem 3 with unit tests generated by the IA

The results obtained with Gemini and ChatGPT are questionable. Did the IA find a general solution that passes all conceivable tests, or did they find a solution that only passes the required tests? We will start over with a solution without PDF to force the IA to go online and search for the information they need. And we modify our instructions as follows:

The text file [instructionsSansPDF4.txt] already contains 14 required tests. To these tests, we add the following instructions:


7 - tu ajouteras autant de tests unitaires que nécessaires pour vérifier les cas limites du calcul de l'impôt.
 
Pour le code tu complèteras le script suivant auquel tu auras rajouté tes propres tests.
 
# =========================
# Unit tests (tolerance ±1 €)
# =========================
 
TESTS = [
     # (adults, children, income) -> (tax, discount, reduction)
    ((2, 2, 55555), (2815, 0, 0)),
    ((2, 2, 50000), (1385, 384, 346)),
    ((2, 3, 50000), (0, 720, 0)),
    ((1, 2, 100000), (19884, 0, 0)),
    ((1, 3, 100000), (16782, 0, 0)),
    ((2, 3, 100000), (9200, 0, 0)),
    ((2, 5, 100000), (4230, 0, 0)),
    ((1, 0, 100000), (22986, 0, 0)),
    ((2, 2, 30000), (0, 0, 0)),
    ((1, 0, 200000), (64211, 0, 0)),
    ((2, 3, 200000), (42843, 0, 0)),
    ((2, 2, 49500), (1297, 431, 324)),
    ((1, 0, 18535), (359, 491, 90)),
    ((2, 0, 43333), (2270, 0, 400)),
]
 
 
def _ok(a, b, tol=1):
    return abs(a - b) <= tol
 
 
def run_tests(verbose: bool = True) -> bool:
    all_ok = True
    for (params, expected) in TESTS:
        a, e, r = params
        exp_impot, exp_decote, exp_reduc = expected
        res = calcul_impot_2019(a, e, r)
        ok_impot = _ok(res.impot, exp_impot)
        ok_decote = _ok(res.decote, exp_decote)
        ok_reduc = _ok(res.reduction, exp_reduc)
        test_ok = ok_impot and ok_decote and ok_reduc
        if verbose:
            print(
                f"Test {params} -> obtenu (impôt={res.impot}, décote={res.decote}, réduction={res.reduction}) | attendu {expected} | {'OK' if test_ok else 'ECHEC'}")
            if not test_ok:
                print(
                    f" Détails tolérance ±1€ : impôt ok? {ok_impot}, décote ok? {ok_decote}, réduction ok? {ok_reduc}")
        all_ok &= test_ok
    if verbose:
        print("\nRésultat global :", "TOUS LES TESTS PASSENT ✅" if all_ok else "AU MOINS UN TEST ÉCHOUE ❌")
    return all_ok
 
 
if __name__ == "__main__":
    run_tests()

Lines 11–24: the 14 required tests;
Lines 5-55: this code comes from the script generated by ChatGPT. We will require Gemini to use this code to facilitate comparisons between the two generated scripts.

We start with ChatGPT:

Its first response is incorrect. I tell it so by providing the execution logs:

Its second response is correct. ChatGPT added the following 11 tests to the 14 required tests:

# Additional boundary cases (bearing edges/rounding)
TESTS += [
     # 10% allowance: floor and ceiling
    ((1, 0, 3000), (0, 0, 0)),  # 10% = 300 < floor 437 => RNI low -> zero tax
    ((1, 0, 200000), (64211, 0, 0)),  # ceiling already covered in initial tests

     # Discount: just below/above thresholds
    ((1, 0, 25000), None),  # diagnostic
    ((2, 0, 35000), None),  # diagnostic

     # 20% reduction: full entitlement vs. capping
    ((1, 0, 17000), None),  # diagnostic
    ((2, 0, 34000), None),  # diagnostic
    ((1, 0, 20000), None),  # diagnostic
    ((2, 0, 40000), None),  # diagnostic

     # Change of units (capping QF)
    ((2, 1, 80000), None),
    ((2, 2, 80000), None),
    ((2, 3, 80000), None),
]

There are now 25 unit tests. I manually verified the 11 new tests using the official DGIP simulator, and they pass.

Now, let’s move on to Gemini. This is going to be much more complicated. It will manage to generate a script that passes all 25 tests for ChatGPT, but only after debugging long.

Below is the debug list:

Strangely, a majority of the tests failed, even among the 14 required ones, whereas in the past Gemini had generated code that passed them all.

The following response from Gemini is still incorrect:

Nor is the following response:

Nor is the following response. So I’m changing my approach. I’m asking it to pass the 25 tests that ChatGPT passed, attaching the logs from ChatGPT:

Gemini fails. It did add the tests from ChatGPT. I attach the logs from its execution:

Still no:

Still not working:

Still not working, but it’s better:

Gemini is making new errors:

It’s improving again:

This time, it’s right:

Undoubtedly, in this specific example of calculating the 2019 tax using the constraints specified in the instruction file, ChatGPT was more accurate than Gemini. But this is just one example.

We can go further. We can ask Gemini to regenerate a PDF file based on the calculation rules it used to pass the 25 tests. We want to see if it has changed its initial reasoning regarding the calculations for the discount and the 20% reduction:

This time, Gemini generated a MarkDown file, which I then converted to PDF and [Le problème selon Gemini version 2]. And Gemini has indeed changed its reasoning:

We can see that the specific discount calculation and the tiebreaker rule are no longer present. Gemini has now adopted the logic of ChatGPT.