Certified Tester Testing with Generative AI v1.0

A Generative AI model is being used to generate automated test scripts. The test manager wants to assess whether the generated test scripts are usable.
Which of the following metrics is MOST appropriate for evaluating this output?

A tester finds that the Generative AI model is consistently generating test cases with insufficient detail and lacking certain critical test steps that are present in the input requirements.
Which prompt refinement technique would MOST directly help address this issue?

In the context of Generative AI systems, which definition CORRECTLY describes a reasoning error?

A tester uses an LLM to prioritize a set of regression test cases for a critical banking application. The LLM is given detailed information, including the criticality of associated modules, historical defect rates, and inter-dependencies between test cases (e.g., Test B can only run after Test A completes). The LLM is asked to generate a prioritized list for the next sprint's execution.
Which of the following LLM-generated prioritization statements indicates a reasoning error?

To reduce the likelihood of hallucinations and reasoning errors when using an LLM for test tasks, which mitigation technique is MOST effective?

Which technique is used to reduce variability and promote more consistent outputs from Large Language Models, particularly for long generated texts?

Which of the following represents a VALID data privacy concern when using Generative AI in software testing?

An attacker specifically designs prompts that are too long for the LLM's context window, intending to overwhelm its memory and inadvertently expose confidential details from its training data.
Which type of attack vector does this scenario BEST describe?

Match each type of attack vector against a Generative AI model (1-4) with its corresponding example (A-D) in the context of software testing:

Attack Vector -
1. Data exfiltration
2. Request manipulation
3. Data poisoning
4. Malicious code generation
A) An attacker provides a highly crafted input to an LLM, causing it to inadvertently reveal snippets of confidential training data, such as internal API keys.
B) A malicious tester repeatedly submits inaccurate or misleading evaluations when prompted to rate the quality of an AI-generated test report, thereby degrading the model's future performance.
C) A user intentionally includes a subtly altered image in a multi modal prompt, causing the LLM to misinterpret the context and generate irrelevant or incorrect acceptance criteria for a test task.
D) An adversary designs a prompt to coerce the LLM into generating executable code that, when run, attempts to establish an unauthorized communication channel to an external, controlled server.

When an organization implements Generative AI for software testing, which mitigation strategy is MOST crucial for protecting data privacy?

A software testing team uses Generative AI for:
- Task A: Generating short text summaries and analyzing test logs
- Task B: Creating synthetic UI images and multimodal test data
Considering AI energy use, which statement BEST reflects the environmental impact of these tasks when widely adopted?

Which of the following is a legally binding regulatory framework for Artificial Intelligence?

In an LLM-powered test infrastructure, which component is primarily responsible for serving as the user interface where testers interact with the system by inputting queries or commands?

In a Retrieval-Augmented Generation (RAG) system, what is the primary purpose of storing document chunks as high-dimensional vectors (embeddings) in a vector database?