Direct shared link works for users outside org, but embed with same link fails with 401

My shared links ("People with the link  Can view and download") are working when accessed directly by someone without a box account or who is not part of our organization, but when I embed the link on our site and try to access it as a non-box/non-org user, the embed fails with a 401. Why does the link work but not the embed (same embed works for a box user so the embed is structured correctly).