How I created my own personal AI Hiring Assistant in 2 days

Posted onMarch 13, 2025 Edit on by t.w.jubb

This article summarises the architecture and steps to create the “CV Assistant” feature on my webpage, this is a frontier LLM model which has access to my CV and experience and can reply in real time to questions.

Instead of trawling through a long CV, or skimming to try and pick out buzz words; why not simply ask my assistant the questions you want answers to?

This tutorial will guide you through how to do the same. It will cost you around $4.00 per month to maintain because you need an IP address for the app, assuming you already own a domain.

Introduction

This tutorial requires a grasp of python, AWS and the basics of webpages/networking and as such is not taken from complete beginnings (as I often like to do) as it would be too long. Additionally this architecture costs around $4.00/month due to the use of cloud services and public internet.

In my case, it adds genuine value to my website and so these costs are acceptable, but this is not an appropriate project unless you consider the costs bring you some benefit above learning the tools. Having said that, the costs are incurred cumulatively, so if you wanted to try this out for a day it would cost very little.

The costs are as follows

  • Domain : you need a domain (mine is www.thomasjubb.com), usually hosting your own website. You can buy these from many online domain hosting platforms (such as Bluehost, ~£35 per year)
  • AWS : There is no cost for setting up an AWS account but we will use an elastic IP which costs $3.60 dollars monthly if in permanent use. Small costs can be incurred from EC2.
  • openAI : We will use openAI’s API which is billed per question. You need to pay $5.00 upfront to access the API, but queries are billed against this and the total cost of querying is very minimal. It would take a long time (or very high demand) to spend this.

To summarise : the monthly cost is around $4.0 (dominated by the cost of a public IP address); but you need $5 upfront for openAI.

This guide is for macOS (I would love to add Windows/Linux at a later date).

Step 1 – Setup

1.1 Configure the Environment

  • Install miniconda for python environments. Instructions here.
  • [Optional] Install PyCharm (community) for development. Download here.
  • Create accounts for github, AWS and openAI
  • Clone the repository 
git clone https://github.com/TWJubb-CB/HiringManagerGPT.git

1.2 Configure AWS

You need an AWS account. Setting one up can be a little laborious; but try to follow best practice for security, creating a user account for working on projects and not billing directly from your root account.

  • You also need the AWS CLI installed in the terminal (follow these steps); confirm installation with aws --version returning a version number.
  • You then need to set up an access key for using AWS from the CLI, you can do this from the security menu in the AWS console (see here for details). Keep a temporary copy of your key ID and secret

We then need to configure a profile from the terminal, I’ve called mine ‘cvgpt’

aws configure --profile cvgpt

This will request some information; enter the access key you set up before (ID and secret) and select a region closest to you (for me it’s eu-west2, London). Now set an environment variable to use the new profile with all subsequent aws commands

export AWS_PROFILE=cvgpt

1.3 Configure openAI

  • Set up an openAI account
  • Sign up for the API here, and add £5.00 funds as required.
  • Set a cost limit of £5.00/month

1.4 Configure Terraform

Install homebrew if you haven’t already; follow these instructions

  • Next, install tfenv with homebrew with brew install tfenv
  • Using tfenv, install the latest version of terraform as below
# install the latest version of terraform (check website)
tfenv install 1.11.1

#; view installed terraform versions
tfenv list
# * 1.11.1 (set by /opt/homebrew/Cellar/tfenv/3.0.0/version)

# set the active terraform version
tfenv use 1.11.1

1.4 Setup python environment

Assuming you have installed miniconda, we create an environment called gradio_env

conda create --name gradio_env python=3.10
conda actiavte gradio_env
pip install gradio openai dotenv tiktoken boto3 cryptography "urllib3<2"

Each time you open a terminal make sure to conda activate gradio_env

Step 2 – Cloud Compute (EC2)

The first step is to set up the cloud infrastructure which will host our app. Best practice would probably be to use Docker/Kubernetes but here we are just going to go straight to EC2 because the infrastructure we need is very light and the steps are more instructive.

2.1 Elastic IP

For reasons that will be explained later, we need an elastic IP. An elastic IP is a fixed public IP address; when we provision an EC2 instance without it, a random public IP address will be assigned which makes other steps much harder. The pricing of an elastic IP is the same as a random public IP (the difference is that you will be charged for the elastic IP per hour whether in use or not, so remember to destroy it)

WARNING : Once you create an elastic IP, it will be charged at $0.05 per hour until it is destroyed. Don’t forget to remove it once you are finished with your app.

  • Manually allocate an elastic IP through the AWS console (note that you cannot have more than 5 of these and they cost $3.60 per month so handle with care).
  • Write down the “allocation_id” of this elastic IP

2.2 SSH Key

We are going to have to do some setup on the EC2 instance ourselves, which we means we need SSH (and SCP for transferring files) access (even if we automate this setup something will still need SSH access). For this, we create a key pair called `test-ec2-key` to allow us to SSH into the EC2 securely:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/test-ec2-key

This creates two files

~/.ssh/test-ec2-key     (private key)
~/.ssh/test-ec2-key.pub (public key)

2.3 Terraform Setup

Now we are ready to create our AWS infrastructure. This can all be done (a) from the AWS browser console manually (b) programmatically from python using boto3 or (c) from an IaC (Infrastructure as Code) tool.

In this case I advocate for option (c) because although there is a learning curve, provisioning cloud infrastructure using these tools vastly simplifies the whole process once you’ve set it up for the first time; and avoids errors from manual steps.

My go to tool is terraform. This can be installed on mac using homebrew (see section 1.4) using tfenv which is very convenient for managing different terraform versions

# move to the infra folder 
cd /REPO/infra

tfenv use 1.11.1

Because you have already set up AWS via aws configure; you should be authenticated and terraform will use these credentials to manage infrastructure for you.

All of the infrastructure is created from the main.tf file. You will also see the variables.tf file, which collects some of the variables that are user-specific.

To set the user specific variables, create a file inside infra/ called terraform.tfvars, and add the following

instance_type    = "t2.micro"  
aws_region       = "eu-west-2"  
elastic_ip_id    = "eipalloc-XXXXXXXXX"  
ec2_keypair_name = "test-ec2-key"
conda_env_name   = "gradio_env"
  • aws_region : set this to your preferred (closest) region, which should be consistent with your AWS profile from section (1.2)
  • instance_type : you can leave this at t2.micro (computation load is very low)
  • elastic_ip_id : Set this to the ID value you generated in (2.1)
  • ec2_keypair_name : the name of the keypair from (2.2) (“test-ec2-key” above)
  • conda_env_name : The name of the conda environment from step (1.4)

2.4 Terraform Execution

Terraform works via a series of commands, run from the infra folder (the folder containing main.tf).

The basic use of terraform is via the init, plan, apply and destroy commands

cd ./infra

# initialise terraform, create a bunch of temp files to manage resources
terraform init

# plan the deployment of resources on AWS
terraform plan

# actually provision resources on AWS
terraform apply

The latter command will ask you to type ‘yes’ to confirm. Once you do this, terraform will go about provisioning all the requested resources on AWS for you (they will begin charging once created).

WARNING : terraform apply will provision resource on AWS that start billing.

This often goes wrong as you develop, or maybe you just want to update the code and re-provision. Make sure you terraform destroy before changing any code, so that terraform can correctly remove all the resources you created. You can always double check in the AWS console as all resources will be listed there and can be manually destroyed if you’re in a tangle.

If everything goes to plan, an EC2 instance will be created with SSH access. You can check the instance exists and is running in the AWS console; but we will connect to it from a terminal (replace the name of the keypair and IP address below to match, using your fixed elastic IP)

#             -[replace]--        --[replace]-
ssh -i ~/.ssh/test-ec2-key ubuntu@18.XXX.XX.XX

This instance can act as a sandbox linux machine that we can use to test installation instructions and similar. We have now successfully created the server machinery on the cloud.

There is a bash script infra/user_data.sh that will be run on the EC2 instance when it boots; which installs what we need. You can actually SSH into the instance while the script is running; therefore be careful to wait enough time for all the commands to complete (a few minutes)

You can monitor the script progress via

# check the progress of the user data init
cat /var/log/cloud-init-output.log

And watch out for the final message that indicates the script complete ok. This also serves as a key debugging step when you are testing/writing the user_data.sh.

Step 3 – Create the Chat App (and run locally)

3.1 Set up the LLM

In the app/data/ folder you should see two text files. These must be written for your use case

  • Add your CV data to ./data/cv.txt in plain text or markdown format (this is the information that the LLM will draw on to answer questions, add as much detail as you can). Note that the more information you add, the more each query will cost so try to keep it concise and use bullet points.
  • Edit the ./data/system_prompt.txt to reflect your name and contact details (this is the system prompt for the LLM)

Create a .env file in the root of the repository and add the name and the API key as follows. Make sure you use the API key for openAI you generated in step (1.3)

# .env
TF_VAR_OPENAI_API_KEY=sk-proj-xxxxxxxxx
TF_VAR_OPENAI_API_KEY_NAME=OpenAI_Key

You can use whichever name you wish

3.2 Run the App locally to test

Rather than go step by step through the building of the app, take a look at the chat_app.py file which contains all of the code. Typically, good code uses OOP (classes), but given that this app is very simple I think it’s best to leave it with a plain python structure.

I added a logging capability, to log all queries and responses; this helps to retain data for you to browse to check (a) what others are asking and (b) how well the bot responds. This creates a maximum of 1 Gb data before overwriting the oldest, to avoid saturating the capacity of the instance. These logs will be placed in app/logs/ named according to their creation datetime.

We will test the app locally first. There are two variables in the chat_app.py file: LOCAL_API_KEY and USE_HTTPS.

  • LOCAL_API_KEY : If true will read the API key from the .env file. If false will read from AWS (requires AWS configured and the terraform script applied)
  • USE_HTTPS : If false will serve over http, if true will serve over https. Make sure to follow the instructions inSection (5.1) for https.

Set LOCAL_API_KEY to True. Recall we created a python environment called gradio_env in step (1.4), we will need that to run the script

conda actiavte gradio_env

cd ./app
python chat_app.py

This should produce a link to http://0.0.0.0:7860 which serves the gradio app locally. Check that this works by writing a test question and hopefully generating a response.

Now that the gradio UI is running, you should confirm the chat actually works (if not, the API key may be the culprit); and you can make changes to the code (change the style, format, etc).

Step 4 – Running the App on the Cloud

In this step, I just want to explain some design choices. You don’t need to do anything for Section (4.1) and (4.2).

We now have a working UI, it runs locally and we can get results from it. We have a couple of problems to solve

  1. Our OpenAI API key must be kept private, in the wrong hands this could be used to charge to your credit card. Make sure you have a limit set on the API on your account and your account login in secured with MFA in the unlikely event that this happens.
  2. We need to serve the gradio App from the EC2 instance so we can access it over the internet.

4.1 Storing Secret Keys : SSM

Let’s try to avoid storing the openAI API key on the EC2 instance; and instead have it ingested into the EC2 instance as needed. This also allows us to the use the API key across multiple apps in the future.

We also wish to avoid costs, so we avoid AWS secrets and KMS, each charge a flat rate per key per month($0.40 and $1.0 respectively). Costs aside, AWS KMS is the best option for securely storing our API key; but we can achieve a high level of security another way

AWS SSM Parameter Store can be used to store strings. The problem is that it stores the string without encryption and it can be viewed in the AWS console; to solve this problem we will store an encrypted version of the API key in SSM, and keep the encryption key in an environment variable on the EC2 instance and locally. Does this really make things more secure? I think it does but of course with our encryption key a bad actor could still use our API key.

So an encrypted copy of the API key is stored in AWS

  • The EC2 instance can read the encrypted API key, and then decrypt it if given the decryption key.
  • The decryption key is stored in an environment variable in the EC2 instance.
  • This means the API key is never hard coded, but if somebody has access to the EC2 instance via SSH; then they could get hold of the API key. Again, spending limits will curtail any damage done in this unlikely scenario, but if you are more worried then use AWS Secrets manager.

You will see that the user_data.sh copies the environment variables for encryption key and key name into the EC2 instance; and from main.tf you will see that terraform stores the encrypted API key in SSM.

4.2 Moving code to EC2

Now we need to move the code into the EC2 instance; there are some choices.

We could do this from github but we don’t necessarily need the entire repository, just some simple scripts that will not frequently change. Plus we don’t want github credentials in the mix to complicate the authentication further.

Because the codebase is very small with few files, we don’t need a complex way to move it to the EC2, and a simple copy via SCP will work well.

Since we have ssh access to EC2 we could manually copy the files

# change the keypair name and IP address
scp -i ~/.ssh/test-ec2-key -r ./HiringManagerGPT/app ubuntu@18.XX.XXX.XXX:/home/ubuntu/app

But this is also easily done by terraform using a file provisioner as you can see in main.tf

4.3 Run the App on AWS

Now lets check the chat App runs fine on AWS. Ensure you set the variables in the script correctly (LOCAL_API_KEY must be False, and USE_HTTPS must be False )

# ssh into the EC2 instance

#             -[replace]--        --[replace]-
ssh -i ~/.ssh/test-ec2-key ubuntu@18.XXX.XX.XX

cd /home/ubuntu

# edit  USE_HTTPS     = False  
#       LOCAL_API_KEY = False
vim app/chat_app.py

# run the app
python app/chat_app.py

Now if you go to the elastic IP address of the EC2 instance

http://xx.xxx.xx.xx:7860

You should see the UI of our app, and it should work correctly

Step 5 – Securing the App with HTTPS

By now, we have managed to set up the UI, on the cloud, and are able to securely call the openAI API to produce responses to questions. This is pretty close to the final product! But… you may notice that the app is served on http (not https), this lack of security is a problem, we need to to serve on https.

The best method is to serve gradio over http internally within the EC2 instance and then use a reverse proxy (nginx) to route https traffic in/out of the EC2 instance.

5.1 Aside : Serving gradio on HTTPS Locally

This is an aside if you want to serve the gradio app locally over https:// which can be useful for other architectures or for debugging.

We already set up the gradio app locally to test the UI design and openAI API. gradio defaults to serving over http (which in unsecure and generates browser warnings); but it can also serve over https. The difference is the use of encryption which requires an SSL certificate; SSL certificates require certain ‘authorities’ to sign them and are based on domains (not IP addresses)

For testing, we can create local self-signed SSL files (these will still trigger browser security warnings)

mkdir ssl
cd ssl
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes

Then gradio can be launched on https (https://0.0.0.0:7680) by changing the variable USE_HTTPS=True in the chat_app.py file.

5.2 Serving gradio on EC2 using HTTPS

The method outlined in (5.1) could be used equally well on EC2; but it leads to several issues

  • The address of the app will be https://10.10.10.10:7680; the use of an explicit IP and the specific port number is pretty unconventional; what if the IP changes? We really want this app to served on a domain like app.thomasjubb.com
  • The use of self-signed SSL certificates will still raise browser warnings and make the app unsecure; it will not look professional. Proper SSL certificates require a domain.

The real purpose of this app is to deploy it from my own website, which is not hosted on AWS:

  • I own the domain www.thomasjubb.com; a DNS system will route traffic from this domain to a specific IP address
  • In AWS, we can provision an EC2 instance with a fixed public IP address (but no domain, this costs extra)

The solution is to create a subdomain of www.thomasjubb.com (say app.thomasjubb.com); and have this point to the elastic IP via an “A record”. Then, our app can be accessed directly from app.thomasjubb.com.

Again, this emphasises the importance of the elastic IP. Not using an elastic IP would mean that each time we restarted the EC2 instance we would have to reconfigure the routing on our website which can take up to 2 days.

5.3 Configure DNS on your personal webpage

We need to set up an A record. Log in to your personal webpage admin. Each vendor will have their own web interface but all should have DNS record settings.

My website is www.thomasjubb.com, and we will create a subdomain app.thomasjubb.com (the prefix app can be anything you like), that routes to our AWS elastic IP.

For this, we need to add an A-record in our DNS settings. Most domain hosting sites will have an area where you can edit your DNS setting, with the ability to add an A record; point this to your elastic IP

It can take 24 hours for a DNS record change to take effect; for this reason if we are going to rapidly be changing our IP, we have a 24 downtime which is no good at all. This is the reason it makes sense to use an elastic (fixed) IP.

We check if A record works (note the potential delay time, although for me this worked within minutes)

nslookup app.thomasjubb.com

This should return the elastic IP address

Once that is confirmed, then http://app.thomasjubb.com will route to the EC2 instance. You will notice however that this will not display the running chat app. That’s because our chat app address is http://XX.XXX.XX.XX:7680 and the subdomain routes to http://XX.XXX.XX.XX

We need to add a reverse proxy to handle this.

5.4 SSL and nginx

Now that the EC2 instance has a domain name (app.thomasjubb.com) via our webpage’s DNS A records, we can generate an SSL certificate from the EC2 instance, in order to serve the app securely over https.

We can use certbot for this, run the following lines from the EC2 terminal

# this will create SLL files at /etc/letsencrypt/live/app.thomasjubb.com/
sudo certbot --nginx -d app.thomasjubb.com

WARNING : The SSL generation has rate limits (5 per week for repeat domain); so repeatedly re-creating the EC2 instance will eventually prevent you using SSL unless you store that certificate somewhere. The code below shows how to copy the SSL data locally from the EC2 instance

# ssh into EC2
mkdir /home/ubuntu/ssl
sudo cp /etc/letsencrypt/live/app.thomasjubb.com/ /home/ubuntu/ssl

# from local machine
cd /repo/path/
scp -i ~/.ssh/test-ec2-key -r ubuntu@18.XXX.XX.XX:/home/ubuntu/ssl/ ./app/ssl/ec2

Open the default file that contains the nginx config

# allow write access to the nginx config
sudo chmod 777 /etc/nginx/sites-available/default

# edit the nginx config
vim /etc/nginx/sites-available/default

Delete the sections which listen on ports 80 and 443 and replace with the text below. Also replace app.thomasjubb.com with your own subdomain

server {  
    # Redirect HTTP to HTTPS  
    listen 80;  
    server_name app.thomasjubb.com;  
    return 301 https://$host$request_uri; 
}  
  
server {  
    # HTTPS
    listen 443 ssl;  
    server_name app.thomasjubb.com;  
  
    ssl_certificate /etc/letsencrypt/live/app.thomasjubb.com/fullchain.pem;  
    ssl_certificate_key /etc/letsencrypt/live/app.thomasjubb.com/privkey.pem;  
  
    # Allow only requests from www.thomasjubb.com  
    if ($host != "www.thomasjubb.com") {  
        return 403;  
    }  
  
    location / {  
        # forward requests to Gradio app  
        proxy_pass http://127.0.0.1:7860; 
        proxy_set_header Host $host;  
        proxy_set_header X-Real-IP $remote_addr;  
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;  
    }  
}

Now you must restart the nginx server.

sudo systemctl stop nginx

# this command shows all processes using port 80 (http) or 443 (https)
# if you see nginx (or anything else) in the output manually kill each PID 
# sudo skill -9 <PID>
sudo lsof -i :80 -i :443

# perform a test of the config before restarting the server
sudo nginx -t

# once the above command returns an empty list, restart the server
sudo systemctl restart nginx

This should now be serving the nginx server, on the domain https://app.thomasjubb.com over https.

You will see from the reverse proxy config that traffic is routed to port 7860 (the default for gradio). If you try to access the URL https://app.thomasjubb.com you will now have a 502 error (bad gateway) because gradio is not yet running. If you want to text nginx is working, you can comment out the location/ block of the config, restart nginx, and you should see a nginx page with no browser SSL certificate warnings at https://app.thomasjubb.com.

Now our server is set up; we need to actually run gradio. There is a simple bash script to do this in the uploaded files

cd /home/ubuntu/

# run gradio as a daemon process
nohup bash app/run_gradio.sh 2> gradio_error.log &

# check it is running
ps aux | grep run_gradio.sh

# if/when needed : kill the script
pkill -f run_gradio.sh

That’s it! Now if you go to https://app.thomasjubb.com it should load the gradio UI.

One final step is to embed the app on a given webpgae. You can achiev this using an iframe; just add a free html element with the following code

<body>  
    <iframe src="http://app.thomasjubb.com" width="100%" height="600px" frameBorder="0" allowtransparency="true" style="background: #FFFFFF;"></iframe>  
</body>

Step 6 – Ongoing Maintenance

Once this solution is deployed it is pretty low maintenance.

  • You need to keep the terraform state files so you can destroy if needed. But even if you lose them you only need to manually delete the EC2 instance and the SSM parameter store entry (plus the elastic IP if you are removing the app forever). In a full enterprise solution we would maintain the terraform state on S3.
  • Check in with the openAI API dashboard to track API usage and costs.
  • Check in with the AWS billing dashboard from your root account to track the EC2 and elastic IP costs
  • Keep in mind the A record entry in your webpage DNS; if the IP address of your app changes, you need to modify this record.

Summary

This article covers the architecture I used to produce the “CV Assistant” feature of my webpage, allowing visitors to ask questions about my experience and save themselves time trawling my CV or other information.

This solution uses frontier Large Language Models (LLMs), includes a UI served securely over https and is optimised for cost.

Opinion : On one hand, it seems like a lot of work to end up with one little text box and a button on my webpage. On the other hand; to deploy LLMs to add genuine value to a little personal webpage like mine would be space-age not just a few years ago (mainly due to the rapid progress in LLMs).

Next Steps

This is not an enterprise solution, but it could be built into one

  • Use ipv6, release the dependency on ipv4 addresses which are the bulk of the cost.
  • Better code practice : Dockerise all the setup of the linux EC2 instance (or create a custom AMI); use CI/CD like github actions to automate the deployment of the repo into the docker container image.
  • Use Kubernetes to manage a cluster of EC2 instances via EKS (Elastic Kubernetes Cluster).
  • Use terraform to provision the kubernetes cluster, and store its state on AWS S3
  • Use AWS route53 and ELB (Elastic Load Balancer) to route traffic to the cluster to handle large amounts of traffic
  • Use KMS (Key Management Service) to store the API keys securely

Leave a Reply

Your email address will not be published. Required fields are marked *