Prompt Engineer, Information Technology
Overview: Are you excited by the opportunity to shape the way AI systems think, respond, and solve problems? We are seeking a detail‑oriented, curious, and quality‑driven individual to join our team as a Prompt Engineer. In this role, you will refine prompts, evaluate AI behavior, analyze large datasets, identify issues, and make light code adjustments to ensure our AI tools perform reliably for customers every single day. The ideal candidate enjoys problem solving, has strong analytical instincts, can translate business requirements into effective prompts, and feels confident testing and improving AI output. If you thrive in a fast‑paced environment, appreciate structured workflows, and want to grow your career in the rapidly expanding AI operations space, this is the perfect time to join us. Applicants must be authorized to work in the United States on a full‑time basis and must not require sponsorship at any point now or in the future. A DAY IN THE LIFE In this role, you will…
- Test and refine prompts to improve the accuracy, clarity, and consistency of AI generated responses.
- Run AI model evaluations to identify errors, hallucinations, and output inconsistencies.
- Apply analytical reasoning to understand business requirements and translate them into effective prompt structures.
- Work with large datasets to analyze AI behavior trends, evaluation results, and performance metrics.
- Troubleshoot and fix minor code issues (typically in Python) when workflows or automation break.
- Use Git for version control, prompt management, and maintaining clean change history.
- Review task instructions and ensure prompts align with expected business rules.
- Monitor AI agent behavior to ensure performance targets and quality standards are met.
- Document prompt variations, test results, data findings, and recommended improvements.
- Collaborate with team members to escalate technical issues or behavioral anomalies discovered during testing
- Work through assigned daily tasks in the prompt tuning, QA and evaluation queue.
WHO YOU ARE You possess …
- Understanding of what makes a good prompt and how prompt structure influences model behavior.
- Strong analytical skills with the ability to interpret business needs and convert them into structured prompt requirements.
- Attention to detail and the ability to spot subtle issues in AI behavior or data patterns.
- Foundational coding knowledge (Python) to read, understand, and adjust scripts when needed.
- Experience using Git for version control and working in collaborative code or prompt repositories.
- Comfort working with large datasets and the ability to extract insights to guide prompt improvements.
- Curiosity, critical thinking, and a desire to understand why an AI system behaved a certain way.
- Excellent written communication skills, especially when documenting findings or prompt adjustments.
- A willingness to learn new AI tools, frameworks, dashboards, and internal systems.
Responsibilities:
- Evaluate AI outputs using internal quality guidelines and identify areas for improvement.
- Analyze large datasets to identify patterns, recurring issues, and opportunities to refine prompts.
- Tune and rewrite prompts to enhance context, reduce hallucinations, and improve accuracy.
- Translate business requirements and feedback into prompt specifications and structured updates.
- Perform basic debugging when prompt output or code errors affect AI workflows or evaluation scripts.
- Maintain prompt libraries, templates, Git repositories, and versioning documentation.
- Report model behavior trends and support root cause analysis for recurring issues.
- Follow operational procedures, testing protocols, and workflow requirements.
- Meet daily productivity and quality benchmarks set by management.
- Collaborate with engineering or product teams when technical escalations are required.
- Perform all other duties as assigned by leadership.
- All other duties as assigned
Qualifications:
- Associate or bachelor’s degree in Business, Data Science, AI, Computer Science, or a related field. Advanced coursework or a Master’s degree is a plus.
- Demonstrated ability to communicate clearly and professionally in written form.
- Detail‑oriented, organized, and able to follow structured testing processes.
- Basic experience with Python, data manipulation, and scripting to write or debug code as necessary.
- Experience with Git or similar version control tools.
- Ability to work with large structured and unstructured datasets.
- Familiarity with generative AI tools (ChatGPT, Claude, Gemini, etc.) is required.
- Comfort learning new systems, internal dashboards, and prompt management tools.