Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [adversarial-attacks]

For questions about the concept of an adversarial attack in machine learning.

12 questions
5
votes
3 answers

What is an adversarial attack?

I'm reading this really interesting article CycleGAN, a Master of Steganography. I understand everything up until this paragraph: we may view the CycleGAN training procedure as continually mounting an adversarial attack on $G$, by optimizing a…
Cyclist
  • 51
  • 1
3
votes
0 answers

Why do adversarial attacks work on CNNs if they classify images as humans do?

A common illustration on how CNN works is as follows: https://www.researchgate.net/figure/Learned-features-from-a-Convolutional-Neural-Network_fig1_319253577. It seems to suggest that CNN in particular classifies images in a similar manner as human…
Sam
  • 205
  • 1
  • 5
2
votes
1 answer

Why do adversarial attack transfer well?

I have read (*) that a common technique to attack a black box AI system based on a neural network is to use it to train a surrogate model to make the same classifications as the black box one. Once this is done, one can look for adversarial examples…
Weier
  • 131
  • 4
2
votes
1 answer

Adversarial attacks on AI systems

I am working on an AI system that classifies images of cats and dogs. I am concerned about the possibility of adversarial attacks, where an attacker can make small changes to an image to fool the AI system into misclassifying it. What are some…
Kanan Suleyman
  • 33
  • 4
2
votes
0 answers

What are the specific differences between vision transformers variants?

I have tried 4 different types of attacks on vision transformers (ViT small and tiny, DeiT small and tiny) but the attack successes on smaller versions are higher than the tiny versions. My understanding is that the smaller versions are stronger…
Craving_gold
  • 31
  • 2
1
vote
1 answer

What constitutes a 'backdoor' attack in machine learning models?

I've recently come across the term "backdoor attack" in the context of machine learning and I'm trying to understand its precise definition and characteristics. From what I gather, backdoor attacks involve embedding malicious behavior or…
hanugm
  • 4,102
  • 3
  • 29
  • 63
1
vote
1 answer

Do adversarial samples violate the i.i.d. assumption?

I am trying to understand why adversarial attacks work in theory. I have read, that the image is perturbed by a special perturbation $X_{adv}=X_1+p$, but i can't find any reference on that perturbation being an i.i.d. violation by itself. If the…
Jan
  • 361
  • 3
  • 13
1
vote
1 answer

How do you game an automatic trading system by messing with data, as opposed to hacking the algorithm itself?

There was a recent question on adversarial AI applications, which led me to start digging around. Here my interest is not general, but specific: How do you game an automatic trading system by messing with data, as opposed to hacking the algorithm…
DukeZhou
  • 6,209
  • 5
  • 27
  • 54
0
votes
0 answers

Can a prompt injection attack exist without predefined system instructions in a language model?

Prompt injection attacks are typically described as adversarial inputs crafted to override or manipulate a language model’s behavior by exploiting its prompt - following nature. In many examples, these attacks rely on the presence of predefined…
hanugm
  • 4,102
  • 3
  • 29
  • 63
0
votes
1 answer

Can software testers transfer their skills into adversarial testing for AI/LLMS?

My sincere apologies if I am more asking for advice, rather than a solution to a specific problem. However, I am trying to survive in this new world as a software tester. I have been doing test automation for about ten years now and think it's vital…
hungryhippos
  • 101
  • 1
0
votes
1 answer

Why don't people use their own random noise to counter adversarial attacks on computer vision systems?

Why couldn't you take the image an AI is given and apply several different random noise filters to the image and take the democratically most common response and use that for the output of the AI. As it stands adversarial attacks require the noise…
Ethan
  • 103
  • 6
0
votes
0 answers

What is the "attack success rate" of an Adversarial Attack?

For a typical adversarial attack, a sample $x_{0}$ is chosen from a training set belonging to class $C_{i}$ and a transformation $A$ is applied such that the adversarial example $x=A(x_{0})$ would be misclassified as $x \in C_{target}$. The…
VirginieDlpts
  • 1
  • 2