Why "red apple* would trigger safety filter?

#23
by wfen - opened

Does Ideogram make this model unusable intentionally?

are you using natural prompts with ideogram or json format prompts?

Here is what I observed:


  1. Safety filter fires randomly on completely safe prompts

Observation
The built-in safety filter is non-deterministic and false-positives
frequently. The same 100% benign prompt:

  "A red apple on a wooden table, soft natural light"

was submitted 7 times and returned the grey "Image blocked by safety
filter" card 6 times (~86% false-positive rate):

  quality=medium  -> blocked
  quality=high    -> blocked
  run 1           -> blocked
  run 2           -> blocked
  run 3           -> blocked
  run 4           -> PASSED (real image)
  run 5           -> blocked

For each run I spend nearly 1 mins, and this safety filter only shows up at the very end, ensure the user really has spent their inference cost.

Sign up or log in to comment