The Ministry of Defence (MoD) staff were warned before the Afghan data leak not to share information containing hidden tabs, according to documents released by the UK's data regulator.
Last month it emerged that the details of almost 19,000 people who had applied to move to the UK were leaked when an official emailed a spreadsheet that contained a hidden tab with the information.
Documents released by the Information Commissioner's Office (ICO) also show that staff there raised concerns about why the body had not issued a fine to the MoD.
The MoD said they had worked to improve data security, but an ICO spokesperson said the government had not yet done enough to learn the lessons.
According to an ICO memo, guidance in place at the time of the leak showed that the 'MoD was aware of the risks of sharing data and explicitly referenced the need to remove hidden data from datasets.'
Hidden tabs are a common feature in spreadsheet software and make information invisible to the user, but still easily accessible if the settings on a document are changed.
The government estimates that the 2022 leak, which led to an emergency resettlement scheme for people at risk of persecution by the Taliban, will eventually cost around £850m.
A super-injunction granted by the High Court in September 2023 prevented the incident from being reported for almost two years, before the order was lifted last month.
Shortly after the MoD became aware of the data breach in 2023, they informed the ICO. The two bodies held a number of secret meetings over the next two years, and documents published by the regulator reveal some of what was discussed.
Government officials described the leak as likely 'the most expensive email ever sent.' Internal emails also show that ICO staff raised concerns about why the body chose not to independently investigate the MoD or issue a fine.
Data breaches by public bodies must legally be reported to the ICO, which can then decide to investigate and potentially fine the organization responsible.
ICO staff privately discussed the potential 'reputational risk' to the regulator after it chose not to take action against the MoD, despite issuing a £350,000 fine for a much smaller Afghan data breach in 2023.
In an email sent the afternoon before the leak became public, one ICO staff member said their justification for not fining the government was still an 'imperfect answer.'
The documents were published by the ICO earlier this month following a Freedom of Information request which was not submitted by the BBC.
Written notes were forbidden during the secret meetings, but an ICO memo detailing the whole timeline was drawn up after the incident became public just last month.
The memo says the MoD took 'intensive measures to recover and delete data from all identified sources' and 'limit loss of control' after the breach was discovered.
In a private email discussion, one ICO staff member questioned why it was 'taking so long to decide whether to investigate' and said 'if I was a journalist I would ask why has it taken two years to ascertain whether or not to take action.'
Another said the ICO had played a 'significant role' but noted that they had been reliant on the MoD to gather evidence.
Ultimately, the ICO decided against sanctioning the MoD as it didn't want to 'impose additional cost to the taxpayer.'
Recently, the BBC revealed there had been 49 separate data breaches in the past four years at the unit handling relocation applications from Afghans seeking safety in the UK.
An ICO spokesperson indicated that the government had 'not yet done enough' to meet the required pace of changes, calling for assurances that necessary improvements are being made.
In response, an MoD spokesperson stated that steps have been taken to 'improve data security' through better software, training, and employing data experts.
They emphasized collaboration with the ICO during an internal investigation and acceptance of all recommendations to prevent similar incidents in the future.
