#ChatGPT. Everyone’s talking about it. The internet is full of examples of AI shaking up how we ask questions. It's a powerful technology but to get the most out of it there are some things you need to consider. This is going to be a long one, so buckle in. 1/22

Well, let’s first take a step back and look at what ChatGPT actually is. 2/22

It’s an artificial intelligence technology called ‘large language models’ – or LLMs. This means that an algorithm has been trained on a large amount of text. In ChatGPT’s case – the internet – but for could also be scientific research, books, social media posts ... 3/22

The algorithms analyse the relationship between different words and, in turn, create probabilities. By asking it a question, it’ll give you an answer based on the relationships of the words in its model. 4/22

Add machine learning into the mix and the result is a chatbot that can answer complex questions in a recognisable, human-like way. 5/22

Struggling to see the downside? Well, using such a large volume of data means it’s not possible to filter what’s accurate and inaccurate; offensive or acceptable. 6/22

And as a result, it does get things wrong. And it’s often those examples that have made the press. Things like hallucinating incorrect facts, being coaxed into creating toxic content or being biased or gullible towards particular arguments. 7/22

So, will it reveal information I type in? A common worry is that LLMs will ‘learn’ from your prompts and offer your information up to other users. There is a concern here, but not for the reason you might think. 8/22

Currently these models don’t give other users your information. But – and this is important – your queries are visible to the organisation that provides the service. This might mean that the provider or its partners are able to read what you’ve typed. 9/22

Queries can be sensitive for different reasons. Perhaps because of the data included within or because of who's asking the question. Imagine if a CEO is discovered to have asked ‘how best to lay off an employee?’ or someone asks revealing health or relationship questions. 10/22

And, as more LLMs spring up, the risk increases that these queries stored online might be hacked, leaked, or accidentally made public. So what does the NCSC recommend? 11/22

Our advice is simple. Don’t include sensitive information in your queries – and don’t submit queries that would cause you grief if they were made public. 12/22

So, is it possible for my organisation to use these tools to automate tasks? While we don’t recommend you use ‘public’ LLMs like ChatGPT for providing sensitive information, ‘cloud-hosted’ or ‘self-hosted’ models might be a better fit. 13/22

If it’s cloud-hosted – check that terms of use and privacy policy. You’ll want to understand how your data is managed, and who it’s available to. 14/22

And if it’s self-hosted – you’ll want to be doing a security assessment. (Time for a plug for our NCSC Principles for the Security of Machine Learning) 15/22

Principles for the security of machine learning

---

https://www.ncsc.gov.uk/collection/mac...

The use of this technology isn’t always well-intentioned. We’ve seen some incredible examples of how LLMs can help write malware. 16/22

LLMs can create convincing-looking results but they’re currently suited to simple tasks. They’re useful for ‘helping experts save time’. But an expert capable of creating highly capable malware is likely to be able to coach LLMs into being able to do it too. 17/22

And LLMs can also be used to advise on technical problems. If a cyber criminal is struggling to escalate privileges or find data, they might ask an LLM and receive an answer that’s not unlike a search engine result, but with more context. 18/22

As LLMs excel at replicating writing styles on demand, there’s a risk of criminals using it to write convincing phishing emails in multiple languages. We might see more convincing phishing attacks, or criminals trying techniques they weren’t familiar with previously. 19/22

So, what does the future look like? It’s an exciting time, but there are risks involved in the use of public LLMs, and individuals and organisations should take great care with the data they choose to submit. 20/22

The NCSC is aware of other emerging threats in relation to cyber security and the adoption of LLMs. We’ll be talking more on the topic in the months to come. 21/22

Read our blog on ChatGPT and the risks posed by large language models. Let us know what you think below! 22/22

ChatGPT and large language models: what's the risk?

---

Do loose prompts* sink ships? Exploring the cyber security issues of ChatGPT and LLMs.

https://www.ncsc.gov.uk/blog-post/chat...