Coinbase has conducted a test to see the usefulness of ChatGPT in performing security analysis for ERC-20 tokens. While it did not hit the desired accuracy, Coinbase notes that it shows potential.
Coinbase is showing more interest in ChatGPT’s application in the crypto industry, with the exchange testing using the AI application. The San Francisco-based exchange conducted a trial for ERC-20 token security analysis, publishing the results in a blog post on March 20.
The test focuses on performing automated token reviews that use Coinbase’s own ERC-20 token review framework. Coinbase notes that “ChatGPT has not clearly demonstrated the accuracy bar that it should be integrated into our asset review process,” although the exchange said it shows potential.
Coinbase compared the results of ChatGPT’s analysis with those of its internal team, which has internal tools for reviewing blockchain projects. The company compared a total of 20 smart contract risk scores between ChatGPT and manual reviews, with ChatGPT yielding identical results 60% of the time. However, of the eight incorrect outputs, the AI labeled a high-risk asset as low-risk five out of ten times.
Regarding specific concerns, Coinbase said that “ChatGPT has not been able to identify when it lacks the context to conduct a robust security analysis.” Other concerns were that the AI is prone to inconsistency and that fact upgrades in ChatGPT may result in output instability.
Coinbase Executive Tested AI First
This is not the first time Coinbase has experimented with ChatGPT. Connor Grogan of Coinbase where did it go Earlier this month, ChatGPT was successful in finding a bug in smart contracts. This is yet another one of the most important and practical uses of AI in the crypto industry.
Grogan offered live Ethereum contracts to ChatGPT, asking it to discover any security vulnerabilities. Not only did it detect exploits, but it also introduced a unique means of exploiting the contract.
However, skeptics say the AI model could have simply searched the internet for the information, as the contract was from 2018. It can have trouble with new unsecured contracts that haven’t been analyzed.
ChatGPT remains the talk of the town
ChatGPT remains the darling of the tech world, with the newly released GPT-4 model driving another wave of hype. It can now process images, which opens up many possibilities for users. Of course, such rapid reforms have raised concerns about the effects on the job market.
However, not everyone seems concerned about the impact ChatGPT will have on jobs. Ethereum developers, speaking at an event in Dubai, did not believe the application would replace developers.
The ChatGPT craze has triggered a rally in AI-based tokens in the crypto market. However, some scams do happen, so users should exercise caution in the market.
disclaimer
BeInCrypto has reached out to the company or the person involved in the story for an official statement regarding the recent development, but has yet to hear back.