Claude cannot control your computer directly, but you can build tools that let it read your screen and run commands through your own code

Claude is a text-based AI made by Anthropic. It cannot open programs, click buttons, or type on your keyboard by itself. However, you can write code that gives Claude information about what is on your screen, and Claude can then tell your code what to do next. This is different from Claude controlling your computer — instead, you are building a bridge between Claude and your computer that you control.

The most practical way to do this is through the Claude API, which lets you send Claude text and images, and receive instructions back. You write a program (in Python, JavaScript, or another language) that takes a screenshot, sends it to Claude, reads Claude's response, and then performs the action Claude suggests. You stay in control the entire time — Claude never runs code directly on your machine.

Key Takeaways

  • Claude works through an API that you call from your own code, not by installing software that controls your computer.
  • You write a program that captures your screen, sends the image to Claude, and executes the actions Claude recommends.
  • You need an Anthropic API key (available at console.anthropic.com) and a programming language like Python to get your free guide.
  • Claude can read screenshots and describe what it sees, but it cannot directly access your files, keyboard, or mouse without code you write to do so.
  • This approach keeps you in control because your code decides which actions to run and which to refuse.

Set up the Claude API and get an API key

Go to console.anthropic.com and sign in with an Anthropic account (create one if you do not have one). Click API keys in the left menu, then click Create Key. Copy the key and store it somewhere safe — you will not see it again. Do not share this key with anyone or paste it into public code repositories.

You will also need to set up billing. Anthropic charges per token (roughly per word) sent to and from Claude. A screenshot and a few back-and-forth messages typically costs less than one cent, but costs add up if you run this many times per day. Check the pricing page at anthropic.com to see current rates.

Save your API key as an environment variable on your computer so your code can find it without you typing it in each time. On Mac or Linux, add this line to your .bash_profile or .zshrc file: export ANTHROPIC_API_KEY="your-key-here". On Windows, search for "environment variables" in Settings and add it there.

Write a basic Python script that sends Claude a screenshot

You will need Python 3.10 or newer. Install the Anthropic Python library by opening a terminal and typing: pip install anthropic. Also install the screenshot library: pip install pillow.

Create a new file called claude_control.py and paste this code:

import anthropic import base64 import subprocess from pathlib import Path client = anthropic.Anthropic() def take_screenshot():     subprocess.run(["screencapture", "-x", "screenshot.png"], check=True)     with open("screenshot.png", "rb") as f:         image_data = base64.standard_b64encode(f.read()).decode("utf-8")     return image_data def ask_claude(image_data, prompt):     message = client.messages.create(         model="claude-3-5-sonnet-20241022",         max_tokens=1024,         messages=[             {                 "role": "user",                 "content": [                     {                         "type": "image",                         "source": {                             "type": "base64",                             "media_type": "image/png",                             "data": image_data,                         },                     },                     {                         "type": "text",                         "text": prompt                     },                 ],             },         ],     )     return message.content[0].text image = take_screenshot() response = ask_claude(image, "What is on the screen? Describe it briefly.") print(response)

Run this script by typing python claude_control.py in your terminal. Claude will describe what it sees on your screen. This is the foundation — Claude is now reading your computer, but not controlling it yet.

Add commands that Claude can suggest and you can approve

The next step is to let Claude suggest actions and have your code carry them out. Modify your script to ask Claude what to do, parse its response, and run only the actions you allow.

For example, you might ask Claude to find a button on the screen and tell you its location. Then your code can use a library like pyautogui to move the mouse and click. Install it with: pip install pyautogui.

Here is a straightforward example: ask Claude where the "Send" button is, and then click it.

import pyautogui image = take_screenshot() response = ask_claude(image, "Where is the Send button? Reply with only the x and y coordinates, like '500 300'.") coords = response.strip().split() x, y = int(coords[0]), int(coords[1]) print(f"Clicking at {x}, {y}") pyautogui.click(x, y)

The key rule: always review Claude's suggestion before executing it. Print the response to the terminal, read it, and only then run the action. Never let Claude's output run directly as code — that is a security risk. Instead, parse Claude's text response and decide which safe actions your code is allowed to perform.

Common mistakes and how to avoid them

The most frequent error is trying to make Claude output code that your script then runs. This is dangerous because Claude might suggest code that deletes files or sends data somewhere you do not want. Instead, ask Claude to describe what it sees and what action to take, then have your code decide whether that action is safe and perform it using a library like pyautogui or subprocess.

Another mistake is not storing your API key securely. Never hardcode it into your script or commit it to a public repository. Always use environment variables or a local config file that you do not share.

Screenshots can also be slow if you take them too often. If you are running a loop, take a screenshot only when you need new information, not on every iteration. Also, large screenshots cost more to send to Claude — consider cropping to just the area you care about.

Finally, Claude's coordinate guesses are sometimes off by a few pixels, especially on high-resolution screens. For critical actions, add a confirmation step or use more reliable methods like searching for text on the screen using OCR (optical character recognition) libraries like pytesseract.

Alternatives: using Claude with existing automation tools

If you do not want to write Python code, you can use Claude through other platforms that already have computer control built in. Some automation tools and no-code platforms have Claude integration, though availability varies. Check the documentation of tools you already use — Zapier, Make, and similar services sometimes offer Claude connections.

You can also use Claude through the web interface at claude.ai and manually follow its suggestions, which is the simplest approach if you only need to do this occasionally. Claude can read images you paste into the chat and suggest steps you can take yourself.

Frequently Asked Questions

Can Claude see my screen without me taking a screenshot?

No. Claude only sees what you send it. You must write code that captures a screenshot and sends it to Claude. Claude cannot access your screen, files, or keyboard without code you write to do so.

Is it safe to let Claude control my computer?

It is safe if you keep control. Never let Claude's output run directly as code. Instead, have Claude describe what it sees and suggest an action, then have your code decide whether that action is safe and perform it. Always review Claude's suggestions before executing them.

How much does it cost to use Claude this way?

Claude API charges per token. A screenshot and a few messages typically costs less than one cent, but costs depend on how often you run it and how many screenshots you send. Check anthropic.com/pricing for current rates and use the API calculator to estimate your costs.

Can I use Claude to control other people's computers?

No. You can only control computers you own or have permission to control. The API key is tied to your account, and you write the code that runs on your own machine. Using this to access someone else's computer without permission is illegal.

What programming languages can I use besides Python?

Anthropic provides official SDKs for Python and JavaScript (Node.js). You can also use Claude through the REST API with any language that can make HTTP requests, including Java, Go, Ruby, and C#. Check the Anthropic documentation for examples in your language.