Non-deterministic Vulnerability Detection Benchmark System [P]
<!-- SC_OFF --><div class="md"><p>I work in firmware adjacent to AI, so not an ML guy exactly, so that's why I've come here. For work we got a bit concerned about Mythos and all the hype made me explore some benchmarking work. I now have this pretty cool benchmark that's about 80% done sitting around and haven't had the time to polish it up and show it off.</p> <p>I was hoping some more AI focused people could check it out, tell me if it's duplicate work, or if it is worth putting some time into
Story Overview
I work in firmware adjacent to AI, so not an ML guy exactly, so that's why I've come here. For work we got a bit concerned about Mythos and all the hype made me explore some benchmarking work. I now have this pretty cool benchmark that's about 80% done sitting around and haven't had the time to polish it up and show it off.
I was hoping some more AI focused people could check it out, tell me if it's duplicate work, or if it is worth putting some time into
Related AI News
Syntactically robust NLI for semantics of imperfectly generated text? [R]
Recommendations for speech annotation tools [D]
