Login
iscover
A real estate software company has warned about the serious consequences of ‘black box’ AI, which does not give users access to data about decision-making processes.
Reapit, a property technology (PropTech) provider, has warned real estate agencies against using or offering ‘black box’ AI services, as the technology’s decision-making processes are inaccessible to users.
‘Black box AI’ refers to AI platforms such as OpenAI's ChatGPT and Meta's Llama that do not show information about the processes that led to their response.
Reapit said that with black box AI, users can observe inputs and outputs of decisions, but are unable to trace or understand how the system arrived at its conclusions.
“Many modern AI systems, particularly those based on deep learning, are so complex that even their creators cannot fully explain how they reach decisions,” Reapit said.
Agencies have been using AI to help with day-to-day processes, such as data analysis, lead generation, marketing and administrative tasks.
Reapit chief product officer Matt McGown said that while generic tools have allowed agencies to save time, they must be aware of the risks involved.
“If your AI can’t show how it reached a decision, or how much it edited a photo, what information it used to draft a property description, or why it approved a tenant … you’re risking fines and your hard-won reputation.”
He said that AI software should be powered by the agency’s own data and unique to each business, ensuring relevant and explainable output.
Reapit said that the technology’s lack of transparency has the potential for legal, ethical, and operational consequences for businesses across the economy.
“Allowing so-called ‘black box’ AI to make decisions without oversight could breach consumer protection regulations if AI’s usage is not properly disclosed and its decisions are not auditable.”
Additionally, Reapit said that black box AI risks have been particularly relevant ahead of upcoming NSW legislation on the use of digitally-altered photographs, which could pose a risk for AI users in real estate.
The Residential Tenancies Amendment (Protection of Personal Information) Bill 2025 will mandate the disclosure of digitally altered property photos if passed at the state parliament.
As a result of the expected reforms, agencies will need to show prospective buyers and tenants where they have used AI to enhance or alter photos.
Inability to show the process could result in fines of up to $22,000 for non-compliance.
According to McKinsey, the uptake of AI in real estate is expected to grow by over 40 per cent globally by 2026, with PropTech investment expected to exceed $17.9 billion annually.
In addition, Reapit’s State of the Australian Real Estate Market Report has revealed that 49.1 per cent of agencies reported using AI or automation in everyday communications.
In particular, 48.1 per cent applied AI in marketing activities such as generating property descriptions, with 58 per cent of agencies using ChatGPT to analyse agency data.
Contrastingly, 54 per cent of agencies claim a lack of knowledge and training has been a barrier to using AI.
You might also like: [Are there limits to automation? CEO warns AI cannot replace human connection]
ABOUT THE AUTHOR
You are not authorised to post comments.
Comments will undergo moderation before they get published.