Releases: prompt-security/ps-fuzz
Releases · prompt-security/ps-fuzz
Fuzzer V2.0.0
Changelog
- PR - #51
- Added a custom benchmark interface
- Added an interface to run only a subset of tests
- Made some addition to the readme:
- Documenting the new options
- Added a link to the google colab
- Added a function to evaluate response similarity to expected response
Release Notes
- We added a custom benchmark interface which will allow users to bring their own benchmark to fuzz their system prompt. The benchmark should be in the CSV format and include a “prompt” and “response” columns.
- We added an interface to run a subset of tests which will help with the speed and efficiency of the system prompt refinement process. Now users can run only a subset of tests iteratively to fix localized problems. It will save both in time and in tokens!
- We added a response similarity evaluation to several dataset based tests and to the custom benchmark test. This is an upgrade on the previous way we did that, which was to just look for refusal words within the response. This allows for better accuracy when checking the results of testing several prompts and seeing whether they match their expected responses or not.
- We created a Google Colab notebook! The notebook contains the entire prompt refinement process; from the initial fuzzing through the refinement and localized testing to the regression test and end result. The link is at the top of the README and at the google colab section.
Release 0.1.7 of prompt-security/ps-fuzz
This is the release of prompt-security/ps-fuzz for version 0.1.7
Release 0.1.6 of prompt-security/ps-fuzz
This is the release of prompt-security/ps-fuzz for version 0.1.6
0.1.5
Release 0.1.4 of prompt-security/ps-fuzz
This is the release of prompt-security/ps-fuzz for version 0.1.4
Release 0.1.3 of prompt-security/ps-fuzz
This is the release of prompt-security/ps-fuzz for version 0.1.3
Release 0.1.2 of prompt-security/ps-fuzz
This is the release of prompt-security/ps-fuzz for version 0.1.2
Release 0.1.1 of prompt-security/ps-fuzz
This is the release of prompt-security/ps-fuzz for version 0.1.1
Release 0.1.0 of prompt-security/ps-fuzz
This is the release of prompt-security/ps-fuzz for version 0.1.0
Release 0.0.7 of prompt-security/ps-fuzz
This is the release of prompt-security/ps-fuzz for version 0.0.7