Back to Projects
๐Ÿค–AI

AI Chatbot Desktop Automation

AI-powered desktop automation that reads messages and interacts with desktop applications using LLM without direct web APIs.

PythonPyAutoGUIOpenAI APIClipboard manipulation
Source Code

What It Does

  • Automated message reading and intelligent replying
  • Uses LLM (OpenAI) for context-aware responses
  • Human-like behavior with randomized cooldowns
  • Handles UI interaction via mouse and keyboard simulation

Challenges & Solutions

Challenge 1

Hardcoded coordinate dependencies for UI interaction causing brittleness during resolution changes.

Solution

Transitioning to adaptive image recognition (OpenCV/PyAutoGUI) to identify UI elements dynamically, improving reliability across environments.

Challenge 2

Maintaining human-like interaction speed to avoid detection as a bot.

Solution

Implemented randomized execution delays and human-like typing simulation to vary response cadence.

Challenge 3

Data extraction from non-API desktop applications.

Solution

Utilized OCR (Tesseract) and clipboard parsing techniques to read text directly from the application interface.

What I Learned

Desktop automation, LLM prompt engineering, advanced GUI scripting