Department of Commerce

IT Specialist (AI)

San Francisco, California, Washington, District of Columbia Full time

IT Specialist (AI)

Department: Department of Commerce

Location(s): San Francisco, California, Washington, District of Columbia

Salary Range: $120579 - $195200 Per Year

Job Summary: NIST works with industry and science to advance innovation and improve quality of life. We're looking for a IT Specialist (AI) to join our team! CAISI seeks a candidate to rigorously evaluate AI systems' functionality and trustworthiness in real-world settings, developing scientific methods and tools to assess performance, and identify failure modes. This notice is issued under direct-hire authority to recruit new talent to occupations for which NIST has a severe shortage of candidates.

Major Duties:

  • In this position, you will: Evaluate and assess the functionality and trustworthiness of AI models in compliance with the AI Action Plan. Design methods for scientifically sound evaluations of AI systems. Evaluate AI systems in application, post-deployment, and in real world settings. Develop methods, tooling, and guidelines for rigorous evaluation of end-to-end AI systems in use (e.g., in the workplace and/or in sector-specific tasks) to identify failure modes and areas of un-reliability. Develop analyses of evaluations and AI measurement instruments, themselves, for example evaluation and improvement of red teaming, benchmarks, and automated scoring techniques. Adapt and improve evaluation techniques, and develop new techniques where needed. Collaborate with colleagues and across teams to develop innovative solutions to AI evaluation problems. Communicate complex AI concepts to both technical and non-technical audiences. Inform the development of technical guidelines and best practices for AI evaluation.

Qualifications: All applicants must submit a resume NOT TO EXCEED two pages on standard-size paper. If the resume exceeds two pages, only the first two pages will be reviewed for qualifications or assessment determinations. Basic Requirements: Experience must be IT related; the experience may be demonstrated by paid or unpaid experience and/or completion of specific, intensive training (for example, IT certification), as appropriate. For all positions individuals must have IT-related experience demonstrating each of the four competencies listed below. The employing agency is responsible for identifying the specific level of proficiency required for each competency at each grade level based on the requirements of the position being filled. Attention to Detail - Is thorough when performing work and conscientious about attending to detail. Customer Service - Works with clients and customers (that is, any individuals who use or receive the services or products that your work unit produces, including the general public, individuals who work in the agency, other agencies, or organizations outside the Government) to assess their needs, provide information or assistance, resolve their problems, or satisfy their expectations; knows about available products and services; is committed to providing quality products and services. Oral Communication - Expresses information (for example, ideas or facts) to individuals or groups effectively, taking into account the audience and nature of the information (for example, technical, sensitive, controversial); makes clear and convincing oral presentations; listens to others, attends to nonverbal cues, and responds appropriately. Problem Solving - Identifies problems; determines accuracy and relevance of information; uses sound judgment to generate and evaluate alternatives, and to make recommendations. ZP-2210-IV: In addition to the basic requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-12 level (ZP-III at NIST). Specialized experience is defined as experience demonstrating technical understanding of Artificial Intelligence (AI) generative models and Machine Learning (ML) algorithms; designing scientifically sound technical evaluation methods in real settings; carrying out rigorous technical evaluations to identify functionality and reliability issues; and collaborating and coordinating on AI-related projects. ZP-2210-V: In addition to the above requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-14 level (ZP-IV at NIST). Specialized experience is defined as experience demonstrating advanced technical understanding of Artificial Intelligence (AI) generative models and Machine Learning (ML) algorithms; designing scientifically sound technical evaluation methods in real settings; carrying out rigorous technical evaluations to identify functionality and reliability issues; and leading cross-functional AI-related teams and projects. Establishes best practices with specific technical standards and workflows. Experience refers to paid and unpaid experience, including volunteer work done through National Service programs (e.g., Peace Corps, AmeriCorps) and other organizations (e.g., professional; philanthropic; religious; spiritual; community, student, social). Volunteer work helps build critical competencies, knowledge, and skills and can provide valuable training and experience that translates directly to paid employment. You will receive credit for all qualifying experience, including volunteer experience. The qualification requirements in this vacancy announcement are based on the U.S. Office of Personnel Management (OPM) Qualification Standards Handbook. If requesting reconsideration of your qualification determination, please refer to the following site: Applicant Reconsideration

How to Apply: Please read the entire announcement and all the instructions before you begin an application. To apply and be considered for this position, you must complete all required questionnaires, assessments, and required documentation as specified in the How to Apply and Required Documents section. The complete application package must be submitted by 11:59 PM (ET) on the closing date of the announcement to receive consideration. The application process is as follows: Click the Apply Online button. Answer the questions presented in the application and attach all necessary supporting documentation. Click the Submit Application button prior to 11:59 PM (ET) on the announcement closing date. As part of the application process, you will be asked to respond to four narrative questions. While your responses to these questions is not required and will not be scored, we encourage you to thoughtfully address each question. You will be asked to certify that you are using your own words and did not use a consultant or artificial intelligence (AI) such as a large language model (LLM) like ChatGPT or Copilot. Once your application has been submitted, you will receive a confirmation notification by email. The Human Resources Office will evaluate your application to determine your eligibility for the position. After the evaluation is complete, you will receive another notification regarding the status of your application. To update your application, including supporting documentation: During the announcement open period, return to your USAJOBS account, find your application record, and click Edit my application. This option will no longer be available once the announcement has closed. To view the announcement status or your application status: Click on this: https://www.usajobs.gov/Help/how-to/application/status/. Your application status page is where you can view your application status, USA Hire assessment completion status, and review your notifications sent by the hiring agency regarding your application. Need help applying? If you are experiencing any difficulties with the online application process (anything after you have clicked the 'Apply Online' button), please contact the Agency Contact listed on this announcement between regular business hours (8:00 a.m. - 4:00 p.m. ET). If you receive any kind of error message, please be ready to provide a screenshot or, at a minimum, the error message text and number. We will provide you with a technical support number for further assistance if necessary. If you are having trouble with USAJOBS (searching for jobs, account password/profile issues, or Resume Builder), please visit the USAJOBS Resource Center. We regret that we are unable to support any issues with USAJOBS, as this service is not maintained by the NIST. If you are experiencing a significant hardship that hinders your ability to apply online, please contact the Agency Contact listed on this announcement prior to the vacancy closing date and during regular business hours to discuss your situation.

Application Deadline: 2025-12-29