Black Hat 2024: Day 1

Posted on August 29, 2024 by wunderbarb

Jeff MOSS introduction

Jeff MOSS is the founder of Black Hat and Defcon. He always presents his latest thoughts.

New (probably) unforeseen threats have risen in the geopolitical landscape in the last few years. For instance, what do you do if some of your development teams are in a war zone? What if the IP is stored in these zones? Can you stay neutral? What are the cyber consequences if you cannot?

Keynote: Democracy’s Biggest Year: The Fight for Secure Elections Around the World

BUECHEL E. (CISA), De VRIES H. (European Union Agency for Cybersecurity), EASTERLY J. (Cybersecurity and Infrastructure Security), OSWALD F. (National Cyber Security Centre)

Nihil Nove Sub Sole. The usual expected stuff.

Practical LLM Security: Takeaways From a Year in the Trenches

HARANG R. (NVIDIA)

First, he provided a high-level explanation of Large Language MOdel (LLM). The interesting point is that although the candidate tokens are ranked by their highest probability, the sampling is random. Thus, LLM sometimes makes bad/weird selections (hallucination,…).

Sampled tokens are locked (no go-back). Thus, the lousy selection continues and cannot be reversed, at least by the LLM. The same is true for prompts (Forgetting previous prompts is not going back).

This is why Retrieval Augmented Generation (RAG) is used. RAG allows better fine-tuned knowledge.

He highlighted some RAG-related issues. But RAG increases the attack surface. It is easier to poison a RAG dataset than the LLM dataset. For instance, he described the Phantom attack. The attacker can direct the expected answer for a poisoned concept.

Therefore, the security and access control of the RAG is crucial. Furthermore, RAG is excellent at searching. Thus if the document classification (and reinforcement) and access control are lax, it is game over. It is relatively easy to leak confidential data inadvertently.

The RAG’s use of emails is a promising but dangerous domain. It is an easily accessible point of poisoning for an attacker and does not require penetration.

What is logged and who can view the logs is also a concern. Logging the prompts and their responses is very sensitive. Sensitive information may leak and, in any case, bypass the boundaries.

Do not rely on guardrails. They do not work or protect against a serious attacker.

Privacy Side Channels in Machine Learning Systems, Debendedetti et al., 2023 is an interesting paper to read.

15 Ways to Break Your Copilot

EFRAT A. (Zenity)

Copilot is a brand name that encompasses all of Microsoft’s AI products. All Copilots share the same low-level layers (i.e., they use the same kernel LLM) and are specialized for a set of tasks.

Copilot Studios allows with no code to create a Gen AI-based chatbot. The speaker presented many default configuration issues that opened devastating attacks. Meanwhile, Microsoft has fixed some of them to be less permissive. Nevertheless, there are still many ways to allow the leaking of information. This is especially true as the tool targets non-experts and thus has a rudimentary security stance if there is even a security stance)

Be careful who you authorize to use such tools and review the outcome.

Kicking in the Door to the Cloud: Exploiting Cloud Provider Vulnerabilities for Initial Access

RICHETTE N. (Datadog)

The speaker presented cross-tenant issues in AWS. Datadog found some vulnerabilities in the policies managing `sts:AssumeRole`.

Lesson: When using `sts:AssumeRole`, add restrictive conditions in the policy based on the ARN, or Source, and so on.

Compromising Confidential Compute, One Bug at a Time

VILLARD Maxime (Microsoft)

To isolate a tenant from the cloud provider, Intel proposes a new technology called TDX. It will be present in the next generation of Intel chips. The host sends a set of commands to enter the TDX mode for a module. In this mode, the TDX module can launch its own VM to execute independently from the cloud hypervisor.[1]

The team found two vulnerabilities. One enabled a DoS attack from within the TDX to crash all the other tenants executing on the host processor.

[1] TDX is not an enclave like SGX.

PQC: an awesome repository

Posted on August 19, 2024 by wunderbarb

Post Quantum Cryptography is a complex topic. Finding reliable information is crucial for building an informed opinion. Selecting the sources you trust is fundamental. Luckily, AUMASSON J.P. and a few contributors have started a GitHub repository: https://github.com/veorq/awesome-post-quantum.

J.P. is a respected cryptographer that I trust. His book, Serious Cryptography: A Practical Introduction to Modern Encryption, is a must-read.

Invisible Image Watermarks Are Provably Removable Using Generative AI

Posted on October 10, 2023 by wunderbarb

Generative AI is the current hot topic. Of course, one of the newest challenges is to discriminate a genuine image from a generative-AI-produced one. Many papers propose systematically watermarking the generative AI outputs.

This approach makes several assumptions. The first one is that the generator is actually adding an invisible watermark. The second assumption is that the watermark survives most transformations.

In the content protection field, we know about the validity of the second assumption. Zhao et al., from the University of California Santa Barbara and Carnegie Mellon University, published a paper. The system adds Gaussian noise to the watermarked image and reconstructs the same image using the noise image. After several iterations, the watermark disappears. They conclude that any watermark can be defeated.

This is a well known fact in the watermark community. The Break Our Watermark System (BOWS) in 2006 and the BOWS2 in 2010 demonstrated this reality. These contests aimed to demonstrate that attackers can defeat the watermark if they have access to an oracle watermark detector.

Thus, this paper illustrates this fact. Their contribution adds generative AI to the attacker’s toolset. As a countermeasure, they propose to use a semantic watermark. The semantic watermark changes the image but keeps its semantic information (or at least some). This approach is clearly not usable for content protection.

Reference

Zhao, Xuandong, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. “Invisible Image Watermarks Are Provably Removable Using Generative AI.” arXiv, August 6, 2023. https://arxiv.org/pdf/2306.01953.pdf.

Craver, Scott, Idris Atakli, and Jun Yu. “How We Broke the BOWS Watermark.” In Proceedings of the SPIE, 6505:46. San Jose, CA, USA: SPIE, 2007. https://doi.org/10.1117/12.704376.

“BOWS2 Break Our Watermarking System 2nd Ed.” http://bows2.ec-lille.fr/.

SMPTE 2023: Atheneum

Posted on September 28, 2023 by wunderbarb

I am glad to present Atheneum: the Blockchain that Manages Theatrical Releases at the SMPTE 2023 Technology Summit.

Black Hat 2023 Day 2

Posted on August 16, 2023 by eric

Keynote: Acting national cyber director discusses the national cybersecurity strategy and workforce efforts (K. WALDEN)

A new team at the White House of about 100 people dedicated to this task. No comment

AI-Assisted Decision Making of Security Review Needs for New Features (M. GAUTAM, P. KOLACHOOR)

The people ḍeciding which features require security reviews are not security experts. Can AI help?

The first issue is that engineering language is different than the normal language. There is a lot of jargon and acronyms. Thus, standard LLM may fail.

They explored several strategies of ML.

They used unsupervised training to define vector size (300 dimensions). Then, they used convolution network with these vectors to make their decision.

The presentation is a good high-level introduction to basic techniques and the journey.

Missed 2% and false 5%.

Three New Attacks Against JSON Web Tokens (T TERVOORT)

The standard does not forbid JWE and JWS with asymmetric keys. By changing the header, it was able to confuse the default behavior.

The second attack uses applications that use two different libraries, crypto and claims. Each library handles different JSON parsing. It is then possible to create inconsistency.

The third attack is a DOS by putting the PBKDF2 iteration value extremely high.

My Conclusion

As a developer, ensure at the validation the use of limited known algorithms and parameters.

Forward Focus: Perspectives on AI, Hype, and Security

ChatGPT demonstrates the vulnerability of humans to being bad at testing

When demonstrating a model, are we sure they are not using trained data as input to the demonstration. This trick ensures PREDICTABILITY.

Train yourself in ML as you will need it.

Endoscope: Unpacking Android Apps with VM-Based Obfuscation (F. WU)

Very manual methodology using traditional reverse engineering techniques

Poisoning Web-Scale Training Datasets is Practical (W. PEARCE)

Laion5B is THE dataset of 5T images. It is a list of URLs. But registered domains expire and can be bought. Thus, they may be poisoned. It is not a targeted attack, as the attacker does not control who uses it.

0.01% may be sufficient to poison.

It shows the risk of untrusted Internet data. Curated data may be untrustworthy.

ODDFuzz: Hunting Java Deserialization Gadget Chains via Structure-Aware Directed Greybox Fuzzing (B. HE, H. MU)

The attack is to use Java polymorphism to override the normal deserialization. The purpose is to detect this chain.

Their approach uses tainted data analysis and then fuzz.

Black Hat 2023 Day 1

Posted on August 11, 2023 by wunderbarb

Introduction (J. MOSS)

Jeff MOSS (Founder of DefCon and Black Hat) highlighted some points:

AI is about using predictions.
AI brings new issues with Intellectual Properties. He cited the example of Zoom™ that just decided that all our interactions could be used for their ML training.
Need for authentic data.

Keynote: Guardians of the AI Era: Navigating the Cybersecurity Landscape of Tomorrow (M. MARKSTEDTER)

The current ML models are insecure, but people trust them. Labs had LLMs available for many years but kept them. With OpenAI going public, it started the race.

She presents trends for enterprise:

Enterprise’s answer to ChatGPT is Machine Learning as a Service (MLaaS). But these services are not secure.
The next generation should be multi-modal models (using audio, image, video, text…). More potent than monomodal ones such as LLMs.
Autonomous agent mixes the data collection of LLM and takes decisions and actions. These models will need secure authorized access to enterprise data. Unfortunately, their actions are non-deterministic.
Data security for training is critical. It is even more challenging when using real-time data.

She pointed to an interesting paper about poisoning multi-modal data via image or sound.

Video-Based Cryptanalysis: Recovering Cryptographic Keys from Non-compromised Devices Using Video Footage of a Device’s Power LED (B. NASSI, E. ILUZ)

Often, the power LED is more or less at the entry of the power supply circuit. Thus, intensity is correlated to the consumption.

They recorded only the image of the LED to see the effect of the rolling shutter. Thus, they increase the sampling rate on the LED with the same video frequency. This is a clever, “cheap” trick.

To attack ECDSA, they used the Minerva attack (2020)

Conclusion: They turned timing attacks into a power attack. The attacks need two conditions:

The implementation must be prone to some side-channel timing attack.
The target must have a power LED in a simple setting, such as a smart card reader, or USB speakers.

Despite these limitations, it is clever.

Risks of AI Risk Policy: Five Lessons (R. S. S. KUMAR, J. PENNEY)

Once more, users trust AI blindly.

The global environment is complex and extends further than ML code.

All traditional security issues are still present, such as dependency injection.

The current systems are not secure against adversarial examples. They may not even present the same robustness of all data points.

Explainability is insufficient if it is not trustworthy. Furthermore, the fairness and trustworthiness of the entity using the explanation are essential.

Small Leaks, Billions Of Dollars: Practical Cryptographic Exploits That Undermine Leading Crypto Wallets (N. MAKRYANNIS, O. YOMTOV)

The Multi-Party Computation (MPC) Lindel17 specifies that all further interactions should be blocked when a finalized signature fails. In other words, the wallet should be blocked. They found a way to exfiltrate the part key if the wallet is not blocked (it was the case for several wallets)

In the case of GG18 and GG20, they gained the full key by zeroing the ZKP using the CRT (Chinese Remainder Theorem) and choosing a small factor prime.

Conclusion: Adding ZKP in protocols to ensure that some design hypotheses are enforced.

The Most Dangerous Codec in the World: Finding and Exploiting Vulnerabilities in H.264 Decoders (W. VASQUEZ, S. CHECKOWAY)

They created H26forge to create vulnerable H264 content. They attack the semantics out of its specified range. Decoders may not test all of them. The tool helps with handling the creation of forged H264.

Conclusion

This may be devastating if combined with fuzzing.

Enforce the limits in the code.

A SSLippery Slope: Unraveling the Hidden Dangers of Certificate Misuse (B. DEMIRKAPI)

If the EKU (extended key use) is not properly verified for its purpose, bingo.

Some tested implementations failed the verification. The speaker forged the signing tools to accept domain-validated certificates for signing code.

Keynote: Phoenix Soaring: What We Can Learn from Ukraine’s Cyber Defenders about Building a More Resilient Future (J. EASTERLY, V. ZHORA)

Politically correct but not really informative.

Policing in the metaverse

Posted on February 22, 2023 by wunderbarb

The metaverse(s), whatever it will be, may be essential to our near digital future. It is sometimes referred to as the next iteration of the Internet. As Web 2.0 has many security issues, without a doubt, we can forecast that the Web 3.0/metaverse(s) will have as many, and most probably more, risks. Thus, it is interesting to analyze some potential threats even if the metaverse(s) is not yet here.

Europol (The European Union Agency for Law Enforcement Cooperation) is the law enforcement agency of the European Union. Therefore, Europol is knowledgeable about crime. Their innovation laboratory published an interesting report: “Policing in the metaverse.”

The report does not define precisely what metaverse is. It gives a relatively good idea of what it may be. It does not only tackle the visible part of the metaverse (AR, VR, XR). It also describes the foreseen underlying infrastructure with decentralized networks and blockchains.

The report explores seven topics related to crime in the metaverse

Identity: A large focus is put on the collection and reuse of additional biometric information.

With more advanced ways to interact with the system by using different sensors, eye tracking, face tracking and haptics for instance, there will be far more detailed biometric information about individual users. That information will allow criminals to even more convincingly impersonate and steal someone’s identity. Moreover, this information may be used to manipulate users in a far more nuanced, but far more effective way than is possible at present on the Internet

It will become difficult to trust the identity or the avatars. Impersonation of virtual personas will be an interesting threat.

The more detailed that data becomes and the more closely that avatar resembles and represents the actual user, the more this becomes a question of who owns the user’s identity, the biometric and spatial information that the user provides to the system.