ChatGPT Does Accountests Debits & Credits Test

Want to see if Accountests will work for your firm? Click on the button for a FREE trial test - use it on a candidate or get one of your staff to give it a go and see what they think.  

As part of our on-going quest to see where Chat GPT is at, we got ChatGPT4 to sit our Debits & Credits Test. This test is typically used by employers for positions where there is a need to check that the core accounting understanding of Dr’s and Cr’s is ingrained and intuitive. The test is a quickfire 20 questions to check if a candidate can deduce the journal entry for a given straightforward transaction, or describe a transaction when told what the journal entry is. 

For those of you haven’t tried asking ChatGPT to provide journal entries, it not only provides the entry, but a detailed rationale of its thought processes. Very helpful. 

ChatGPT scored 18/20 in completing the test. The cut score assessed for this test to show intuitive understanding is 15/20. Typically those with many years experience score 19/20 or 20/20 and complete the test in much less than the time allowed. So ChatGPT is as good at thinking in ‘double entry’ as most accountants. 

Where did ChatGPT go wrong? It’s hard to be critical but to be fair any questions it gets wrong can only be through shortcomings in its data set. The first question it got wrong related to a bank overdraft. Whilst correctly identifying that a bank overdraft would be a liability it insisted it would have a debit balance. The second question related to a journal entry Dr Vehicle Expenses / Cr Owners Drawings. ChatGPT correctly identified that a Dr to Vehicle Expenses was an increase in an expense, but seemed stuck on a Cr to Drawings being a withdrawal of cash from the business. After going back twice to point out the error in its logic ChatGPT got the answer right (and gave me a nice apology for getting it wrong). It did tell me it had learned and would not make the same mistake again – and when I asked the same question a week later it got it right!  

I tried a few random questions. When I tried to fix a P&L coding error it got the journal the wrong way round. I challenged it (without giving more information) and it then got it right. 

We are very very close to the point where AI can: 

1. Review the P&L and all its entries, supplier names and attached invoice details 
   
   
2. Consider both the tax and accounting implications of each transaction based on its knowledge of (say) tax deductibility and normal coding 
   
   
3. Compare expenses to previous periods 
   
   
4. Flag any entry it considers may be incorrectly coded, or may require adjustment for tax purposes 
   
   
5. Propose journal entries or re-coding, requiring a human to just approve or not (like reconciling bank transactions in QBO or Xero) 
   
   
6. Propose tax adjustments to populate the tax return and the tax reconciliation, requiring a human to just approve or not 

The first review of a clients P&L information will not be completed by a junior on the team. It will be completed by the bot, requiring only manager review of matters raised and high level review and sense check. At this stage balance sheet reconciliations will be more challenging, until the bot can firstly work its way through the transactions to establish what makes up the closing balance, and then complete the required review to see if its correct, while also considering what is NOT included but should be. A bit more development might be needed. 

This raises 2 important questions – what are the junior accountants going to do, and how will the managers of the future learn their trade.  

In the meantime ChatGPT is available to you staff and clients to act as a second pair of eyes when you’ve got journals to do.

About the Authors

Giles Pearson FCA was a PwC Partner for 18 years before jointly setting up Accountests.  

Steve Evans has a whole career dedicated to enabling employers to attract, recruit, develop and retain talented individuals and teams, with particular expertise in candidate testing and assessment before setting up Accountests.

Accountests deliver the world’s only online suite of annually updated and country-specific technical knowledge tests designed by accountants for accountants and bookkeepers.