Having worked with these tools for five or so months, I see tremendous opportunities for risk, crisis, and security practitioners as long as we are cautious. One example is how much faster I can build exercise materials using these tools: 50-100 times faster by my calculations—quite the difference. However, there are also some pitfalls to keep in mind.

I want to share some ideas for how you can utilize these tools, but writing up the potential use cases is taking me a while, so to get started, here are some considerations before you jump onto the AI bandwagon (or decide that it's all hype). As usual, the answer is somewhere in between.

Limits and Cautions

Let's start with some quick thoughts on limitations/cautions to consider.

Confidentiality - data used in prompts and queries may be assimilated into the model. I'm trying to learn more about the 'firewalling' of models layered on top of company systems. EG, How does Microsoft stop proprietary data from Outlook emails from leaking back into the ChatGPT model? At the moment, I recommend making requests and sending prompts without company information. This is the approach I take with CrisisDojo: risk and scenario prompts use an anonymized company profile (size, geography, and industry) so no identifying information gets passed to the model.

Data sets are fixed - currently, generally, accessible models are trained on data sets fixed at the time the model was trained, meaning that the models do not have access to live data. Live access will likely arrive in the next 90-180 days, but for now, check the results you get if you are trying to work on anything that needs current information.

Hallucination - models can return incorrect but convincing responses if you aren't careful. The more vague the prompt and older the model, the more likely you are to get a generative response and, therefore, more hallucination. Similarly, the models can be precise but inaccurate, meaning the results can be highly detailed and specific, just completely wrong. These challenges heighten the need for a subject-matter expert in the loop before you depend on anything AI generates.

They're easy to lead astray. Remember, the model responds to what you tell it to do, allowing users to overcome whatever guardrails are in place. Every 'I made ChatGPT Wite Terrible Things' article is an example of that. The models aren't objectionable out of the box, but, given enough time and sufficient prompts asking it to be horrible, the model will eventually be horrible. That means 1) you need to check the results before you share them just in case it's gone off track and produced something objectionable, and 2) someone will find a way to overcome the precautions you put in place.

The Four AI Personality Types

One way to think about the models is having one of four personalities corresponding to quadrants of the accuracy/precision grid.

• "The Lunatic": Imprecise, inaccurate. Spouts utter nonsense that's clearly garbage. Occasionally amusing, mostly annoying.

• "The BS Artist": Precise, Inaccurate. This is the most dangerous 'personality' as it spouts convincing garbage full of details and facts, making it seem highly plausible.

• "The Generalist": Imprecise, Accurate. Gives an excellent general response that's lacking in depth and detail. Good for summary tasks and ballpark answers. Fast.

• "The Expert": Precise, Accurate. Spot on, detailed answers with nuance and insight. It needs careful training and thoughtful prompting, which can take more time.

You want to avoid the Lunatic and BS Artist, but you also have to ask yourself which personality is most appropriate for the task at hand. Just like tasking a human, sometimes you want something fast and general, whereas other situations need an expert. Select your models appropriately.

Three Questions to Ask Yourself

Finally, the questions I'd be asking myself are:

• How can these tools help me be more efficient?

• How can these tools help me be more creative?

• How can these tools help with research and analysis?

I'll share specific use cases shortly, but I've found these considerations and questions helpful while building and researching this topic. I hope these help you too.

Here's to moving fast with abundant caution.