Learning AI Governance and Security

“What I cannot create, I do not understand.” - Richard Feynman

This article is not "good" by my definition. The content in this article is valuable and accurate, but it is incomplete. My education in AI Governance and Security is not "complete"; therefore, I feel uncomfortable writing about it. AI Governance and Security is an emerging domain that is changing wildly daily. So, while I don't believe this article is a complete resource for building competency in AI Governance and Security, it's a good resource. It is undoubtedly the most comprehensive resource of this type I've seen online. You will notice that even the Hugging Face Discord server does not (at the time of this writing) have a channel about AI Governance and security. This is a problem.

Who is this article for? This article is designed for individuals implementing AI Governance and Security within their organizations. This article is also intended for individuals wanting to specialize in AI operational security. This article is NOT for people looking for "hacks" or "top 10 tips" for AI security. This article will not be me rehashing information for you; it's a list of resources for you to consume. I have tried to order the articles in a way that I wish I had found so I would better understand AI. This would have saved me significant time in my learning process.

This article is being released because many people have asked for it. I've been fortunate enough to present at conferences on AI Governance and Security over the last 9 months. After each meeting, people asked me, "How did you learn all this?" It was excruciating; I needed to find a consolidated resource to educate people or organizations on structuring AI Governance and Security operations practices. But I never located one. The NIST AI RMF is incomplete and may be harmful, as I outlined in my article: The NIST AI RMF Won’t Help AI Governance.

My philosophy on information security is that you must understand how something works at a fundamental level to secure it. A significant amount of the 1000 hours I've spent on AI thus far is trying to understand how it works. In cybersecurity, we call an offensive professional who only knows how to use prebuilt tools a "script kiddy." We want to be something other than AI Governance and Security script kiddies. We must understand how AI works. AI security practitioners should be able to create simple models and understand the vocabulary and why things work the way they do.

There are excellent resources available from Nvidia to Standford that cover point topics from an academic perspective. But nothing that brought these resources together into an operational framework. I also wanted to release something when I had the privilege of working through these issues in a production environment. I've completed that milestone with what I believe is a measure of success, so I feel more confident in sharing my learnings with the public.

If you're part of the small population that's the intended audience, let's go on a journey together. I will update this article as I learn more so you can use it as an ongoing reference. About 10 people will likely read this article, but that's okay. I didn't write it for the page views. It was more about releasing something to the passionate few than something designed for mass consumption. Mental junk food is terrible for you…

Basics About Your AI Governance and Security Journey

You need to ask yourself: Are you willing to put in 10,000 hours to learn this topic? Andrej Karpathy and Lex Fridman ran a fantastic podcast on Advice for Machine Learning Beginners. Andrej states that you need to start and put in 10,000 hours. If you don't know who Andrej Karpathy is, you should. He held the position of Director of AI at Tesla, OpenAI, etc. Here is Andrej Karpathy’s entry in Wikipedia.

AI Governance and Security will undoubtedly become a subsection of cybersecurity, like application security. It will emerge as a highly detailed cybersecurity subspecialization that requires significant domain knowledge.

AI Governance and Security Prerequisites

You need to know several things before understanding AI Governance and Security. Artificial intelligence is a massive topic that requires domain knowledge to be effective in the field.

Fundamental Linux Knowledge

It would help if you had a basic understanding of Linux. Ubuntu is my preference because it is well-documented. Learning an operating system is beyond the scope of this article. Still, many AI tools favor Linux operating systems. I have two similar laptops, and installing AI packages on Linux (Ubuntu) has been significantly easier than on Windows.

Python Development Knowledge

Note: If you already have a decent handle on Python you can skip this section.

Anyone interested in cyber governance or security should learn Python. If your job is governance-based, processing reports, and performing data analysis, Python will help you. If you're in a more traditional cyber role and are trying to parse system logs, Python will make you more effective.

Why You Need Python

In traditional cyber / infosec, we preoccupy ourselves with firewalls to protect the network, EDR solutions to protect endpoints, and many other tools. AI / Large Language Models are all about data; you need tools to analyze and visualize the results. The term "trust but (attempt to) verify" has never been more accurate than with Artificial Intelligence.

Some may claim that you don't need to understand all these topics to be effective, but I beg to differ. With the fundamental skills listed below, you can independently validate the security information provided by AI operations teams.

You can learn Python while using various AI systems. The sooner you become familiar with these tools, the more prepared you will be once you dive deeper into AI models and data analysis. For those who want to learn Python, here are some of the pivotal resources I used a few years ago to help me.

Python Books

(Optional) Introduction to Python

I recommend this book if you have not been exposed to other programming languages. Python has some nuances around its data types that should be well understood.

Automate the Boring Stuff with Python

This is an excellent resource for people new to Python. It provides practical examples of projects that are helpful for anyone in information security.

Video Resources

100 Days of Code: The Complete Python Pro Bootcamp

This was a good start if you prefer a video format. Many studies have shown that learning through a book is more effective than video. This video provides broad exposure to Python, so there are elements included that will not be directly relevant to cybersecurity or AI.

NumPy, Pandas (The Library Not the Bear), and Matplotlib

NumPy

NumPy is a package for manipulating complex numerical data objects. Its underlying libraries are primarily written in the C programming language for improved performance.

https://numpy.org/

Pandas

The Pandas library is built on top of NumPy. The best way I can explain Pandas to someone who has never heard of it is an in-memory Excel spreadsheet. It allows for high-speed / highly performant data analysis. This is an oversimplification of Pandas, but that's the "explain it to me like I'm 5" version. The main concepts I've used are DataFrames (in memory Excel spreadsheet) and Series (think of it like a column in a spreadsheet).

https://pandas.pydata.org/

Matplotlib

Matplotlib is a Python library that helps create visualizations from data objects created by libraries such as NumPy or Pandas.

https://matplotlib.org/

Google Colab

I discovered that Jupyter notebooks are widely used in data science. Google Colab is a web-based implementation of Jupyter notebooks. It is also helpful because it allows you to access high-end GPUs relatively cheaply. Later resources use Google Colab to teach you how to create AI models.

Welcome to Colab
Practical Introduction to Google Colab for Data Science

Start Using Gen AI (Just Have Fun!)

If you're learning Python, you can do this in parallel, and I recommend using OpenAI to help you learn Python. As you "play" with AI, you'll learn how to ask it better questions. Asking AI questions is known as prompting, and we'll discuss how to become better at prompting shortly.

AI Text Generation

Unless you're living under a rock, you've likely heard about ChatGPT. Currently, ChatGPT is the most potent large language model (LLM). You can access it via a web interface or a RESTful API. If you're familiar with Python or you taught yourself Python, start working with the ChatGPT RESTful API.

My observation is that businesses are more interested in LLMs than image generation. Besides ChatGPT, there are many LLMs; Anthropic is another LLM that is available with an API. Both OpenAI and Anthropic models are closed source.

To access open-source models, you can leverage a site like replicate.com. Replicate can provide you with access to cutting-edge open-source models like Llama 2 Chat.

AI Image Generation

The two main-stream image generation services are Midjourney and OpenAI's DALL-E 3. They each have their pros and cons. The most significant current drawback to Midjourney is that you're limited by the Discord interface. We will spend less time on AI image generation because large language models receive more business attention.

Prompting (Using Gen AI Better!)

We'll begin our introduction to prompting with a blog post from OpenAI: Best practices for prompt engineering. This post aims to learn "How to give clear and effective instructions to OpenAI models." There is some guidance in the blog post about enclosing text in various special case characters such as triple quotes: """ some text """ It was thought that this would be a way to mitigate prompt injection, but this has not proved to be true over time. Still, this is an excellent introductory resource on prompting.

The two following videos from IBM will provide you with additional information on prompt engineering and tuning. High-quality prompts and prompt turning will help the model provide higher-quality responses. The first video, What is Prompt Tuning by IBM, is 9 minutes long, and the second video, 4 Methods of Prompt Engineering by IBM, is 13 minutes long and explores more advanced topics.

When you're ready to advance your prompting knowledge, we recommend promptingguide.ai for detailed instructions on AI prompting. The Prompting Guide website provides additional details about Zero-Shot, Few-Shot, Chain-of-Thought, and other prompting styles. These styles can help you get better answers by forcing the LLM to "reason" through its answers.

General AI Emersion

If you've worked through the content above, you should be a competent AI operator. Suppose you know or have taught yourself some Python. In that case, you've learned how to interface with various AI resources via the web interface and RESTful APIs. At this point, I would play and enjoy the engineering marvel of AI. There's an excellent article titled: The Importance of Learning Through Play. The article is focused on children, but if you enjoy your AI journey, it will feel less like work and more like… well, play.

In the early days of my AI journey, I used Midjourney to create unique artwork for our home. I've also used ChatGPT to make my work emails seem more friendly, as I tend to be terse in email communication. Explore AI for as long as you like; enjoy it.

YouTube AI Educational Resources

I enjoy the following YouTube channels for general AI news, tips, and tricks.

Learning How GenAI Works Under the Hood

IBM Fundamentals playlist on YouTube has an excellent series on AI Fundamentals. The first video in the series contracts when humans or AIs will provide a more reliable answer. You will learn the fundamental elements of AI, which will help you evaluate the proper use cases for AI implementation. This adequate understanding of AI systems will be critical to help you establish AI governance later. The series will also introduce why businesses may want to use AI. Pay special attention to the video "Large Language Models: How Large is Large Enough?". It highlights the importance of understanding your specific use case in selecting an AI model. You'll need to dedicate about 3 hours to consume the entire series. If you're not interested in SIEM technology, you can skip the last video.

After the AI Fundamentals course by IBM, watch Andrej Karpathy's video, Intro to Large Language Models, on YouTube. Andrej's video will discuss many of the same things found in IBM's video series but at a deeper technical level. This will give you a better understanding of how LLMs work, the kinds of tools some AIs can access, and what multimodality is. A critical element you will be exposed to is that even the most respected AI researchers do not fully understand why these LLMs work the way they do. This fact impacts concepts such as "Trustworthy and Explainable AI" in the NIST AI RMF. Finally, in this section, you'll be exposed to your first real security threat to LLMs, the concept of a jailbreak.

(Optional) The Mathematical Rabbit Hole

We need to take a slight detour before we dive deeper into how AI systems, specifically Neural Networks, are created. Mathematics is a heavily used concept within the AI field. Calculus and Linear Algebra are critical for the implementation of neural networks. Using the software libraries available for AI without understanding the underlying math is possible. Still, I need to take a different route. Suppose you didn't take Calculus and Linear Algebra in college or don't remember it. In that case, you'll need to refresh or build your knowledge. This is optional; just as you don't need to understand the algorithms used in cryptography to use the technology, the same is true with AI. You'll see calculus and linear algebra in the following sections.

There are a few things / key terms you really should understand so I’ll leave references to them her:

Into the Belly of the Beast

Everything about AI was simple up to this point. As someone who was primarily a systems engineer with a focus on InfoSec, I never thought I would need a lot of math for my career. I was wrong.

High-Dimensional Space

I didn't understand it when I started my AI journey, but understanding higher dimensional space is very important. Most humans can visualize an object in three dimensions, but what about in 4, 40, or 400 dimensions? Humans don't have a visual frame of reference for this. When we think about something simple, like building "meaning" around words in language, we need many dimensions to identify word relationships. This occurs in higher dimensional space. Google has done an excellent job with this video: A.I. Experiments: Visualizing High-Dimensional Space.

General AI Architecture

One of the best introductory resources for understanding neural networks I've found is the playlist: Neural Networks by Bevan Smith. This playlist will introduce you to concepts such as the forward pass, backpropagation, gradient descent, and epochs. When you deal with backpropagation, you'll eventually reach gradient descent and the chain rule in calculus.

The “Attention Mechanism” in Neural Networks

You'll learn about weights if you complete the Neural Networks series above. To force AI to focus on critical text elements, you use a method named attention. Google has produced a video Attention mechanism: Overview. AI Attention Mechanisms was a breakthrough in the field. I recommend reading the paper Attention is All You Need.

Encoder-Decoder Architecture

In the Attention Mechanism: Overview video, you were introduced to the concept of an encoder and decoder. Google has produced another video, Encoder-decoder architecture: Overview, that provides more information.

AI Tokens

In the Encoder-decoder architecture: Overview video, you were exposed to the concept of an AI token. Tokens are a critical architectural component of large language models. Andrej Karpathy has another beautiful video that thoroughly explains what a token is and writes live the tokenizer that OpenAI used. This video, titled Let’s build the GPT Tokenizer, can be found on YouTube. This video connected many dots for me in my AI journey, including how tokens were created, what byte pair encoding was, and how tokens in a vocabulary related to the context window.

What Do We Do with Tokens?

Now that we understand tokens, what do we do with them? The Google video we referenced, AI Experiments: Visualizing High-Dimensional Space, gave us a clue. We need to relate tokens and words to one another to derive context from how they are used. We do this by converting words or tokens to vectors, numerical data representations based on dimensionality. I will provide a fictional example of two-word vectors using 5 parameters. This is a gross simplification for those deep in AI; I'm trying to illustrate a point.

Words	5 Dimensions and Values
	Royalty	Gender	Ruler	Parent	Crown
King	0.2	0.9	1.0	.2	.7
Queen	0.2	0.1	1.0	.2	.6

The example above is used to help illustrate how you can derive context between 2 words. Most of the dimension values for these two words are similar. The outlier is gender. The closer to a value of 1 in the dimension, the closer to "male" the word is. The closer to 0 in the gender dimension, the more female is assumed. For a gentle introduction to vectors, Computerphile has a video on Vectoring Words. For a deeper dive, Stanford has an excellent Lecture, Word Vector Representation: Word2vec, that covers how this process occurs. I highly recommend watching the Standford video series. If you watch the Stanford lecture, you'll see that human beings aren't looking at all the available words and creating dimensions like I have above. Machine learning builds word or token vectors, creating labeled dimensions in the background. You can't investigate a database somewhere and see these dimension labels; they exist as relationships between words as they are discovered by the AI system.

Building AI Models

If you've made it this far, you're only 24 hours away from achieving understanding. According to Richard Feynman, "What I cannot create, I do not understand." You will not create your own (simple AI model). An excellent course is available on YouTube: : PyTorch for Deep Learning & Machine Learning. I say "course" because this video is 25 hours long. If you've read the content, you should be well-equipped to handle this course.

Introduction to Security and Governance for AI

Now that you have a solid understanding of AI systems, you can dive into AI attacks. The first document I would read is an excellent resource provided by NIST titled Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. This document provides an outstanding introduction to AI attack taxonomies and how attacks occur at various stages of the AI life cycle. It also covers types of attacks such as poisoning, privacy attacks, etc. Additionally, the Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations will provide you with ideas for mitigation techniques to secure your AI systems.

This paper can be a jumping-off point for finding excellent AI security research papers. The University of Cagliari has done exciting research and produced excellent papers, such as Machine Learning Security against Data Poisoning: Are We There Yet?

I would use some simple google searches against arxiv.org. If you use the google search:

site: https://arxiv.org AI attacks

You’ll be presented with a wealth of information on AI attacks.

OWASP Top 10 for LLMs

If you're looking for additional resources on LLM security, I would direct you to the OWASP Top 10 for LLM Applications. This document covers risks from prompt injection and sensitive information disclosure to training and data poisoning. I like that the document provides guidance on what LLM issues are preventable and which are not. For LLM attacks that are not preventable, it gives guidance on how to architect solutions that mitigate risk.

The NIST AI RMF

I’m not a huge fan of the NIST AI RMF on its own. I’ve outlined my reasons for this opinion in the article: The NIST AI RMF Won’t Help AI Governance. In summary, the NIST AI RMF leaves out a tremendous amount of education on fundamental AI concepts that are critical for proper governance. If you’ve made it through the information presented here, you do have that context and information. You will be able to read the NIST AI RMF and understand the implications of your decisions. You will be able to have an intelligent conversation with your AI team, if there is one, in your organization. This is what governance is about, working with your business partners in an intelligent way to achieve business objectives safely.

Closing Thoughts

On my journey, I felt like I was stumbling through the dark. I know this is an insane amount of information to work through. I don't know what that says about me and how I spend my free time, but it's who I am. I wish I had a document like this to have a logical progression to work through. I aim to make this a living document as I continue my journey. I wish you the best of luck if you've made it this far. If you're interested in further conversations about AI, let me know.