Removed From Bing and DuckDuckGo

Glad this issue is finally getting some traction. I've personally given up on trying to solve this problem for my website, as my multiple messages to Bing support over the past year or so either went unanswered or consisted of naive unhelpful standard messages. Google doesn't need this kind of free advertising, especially not from their only alleged competitor.

uh oh, now that they're no longer doing it to conservatives or other groups of people that the tech bubble considers "undesirables", it's a problem.

woah. maybe allowing private corporations to instantly censor whoever they wish with no accountability, wasn't such a good idea.

@anon Deciding, as an organisation, that you do not want to publish something is not censorship.
There are plenty of places like Fox News for the ‘undesirables’ to promulgate their conspiracy nonsense.

A hammer is supposed to help one drive in nails, not have an opinion about what kind of nails it likes. In the same way, a search tool is supposed to search the web, not to have an opinion about what it "wants to publish".

Somehow all this bollocks about publishers "having the right to express an opinion" has corrupted what a search tool is. If they want to do that they should clearly state: our website does not search the web, but provides a curated view of what we want you to think. Many newspapers at least have the honesty to state their bias.

Google certainly doesn't say "we have an opinion, a bias" in their blurb about themselves: https://www.google.com/search/howsearchworks/our-approach/ Instead, they claim they care about providing the "most helpful information", which should be the information people are looking for, not the information Google wants them to see. I presume the other search engines say the same.

This has serious implications. If people believe search tools are objective, but they aren't, one is distorting the understanding of the entire population without them knowing. Democracy cannot exist with such shadowy interference.

Furthermore, the fact that people have been trained to think these search engines are objective means actually objective search engines cannot compete.

For instance, WebTop, a British search tool, was out-competed, and is no longer available. https://web.archive.org/web/20000708023501/http://www.webtop.com/index.html (Xapian is derived from that product.) It's dead, and Google ended up victorious.

The real world has a much wider range of valid opinions than American mainstream media or Fox. Some of us want to see that full diversity of views, not just those that our local propagandists want us to hear. This is unfortunately a lot harder than it used to be... knowing multiple European languages is no longer enough, when it used to be.

Oh so its not just me! My site (volitans-software.com) doesn't show up on bing either, but clearly searching for it has related results. Its maddening!

> Google certainly doesn't say "we have an opinion, a bias"

It's that famous liberal bias that reality just can't get rid of, so if Google tries to rank correct results over misinformation, they automatically get accused of having "an opinion." I guess "we want to rank correct results over incorrect results" is now a political opinion.

> neither do they seem to be fully indexed

But this has always been the case, no? Bing's index is just much smaller than Google's, so if you have a website that isn't super popular and do a site search, you're going to get five times as many results on Google than on Bing.

@ Old Unix Geek

I get where you are coming from and in principle would agree were it not for the way the web has become monetised and the algorithms that underlie most of it.

As you know, engagement is king.

It turns out people are more engaged by controversy than level-headedness.
There is also a fine line between publishing and promoting.

If you publish outlandish and provably false opinions it will often turn out they drive the most engagement and become the most popular - so re algorithms those views get presented as having primacy by virtue of engagement - rather than merit or even sanity.
It's a fine line to tread and missing your step can end up facilitating the promotion of wrong-headedness, worldwide.

@Plume If Daring Fireball is not popular enough to be fully indexed, that’s not a good sign for Bing.

@Plume:

> 'reality has a liberal bias'

I don't source my wisdom from mediocre comedians. One might think that reality has a liberal bias in a time where the pendulum is swinging in a liberal direction. One won't think that when it swings back.

Reality itself is neither liberal nor conservative, nor anything else anyone might wish to project on it. It's just reality.

@Niall:

I don't believe the solution is to change the world to coddle people from what is. The solution is to change the people so they can recognize stupidity themselves. Simply because someone is on the mainstream media, or leading a country, doesn't mean they aren't spewing complete and utter rubbish.

We don't cover the world in leather to prevent our feet from being harmed. Instead we cover our feet in leather. In the same way, we should learn to think critically (few people do), rather than expecting our information sources to be "safe for consumption without critical thought". Such beliefs are very helpful to those trying to create a propagandized population.

Proper education is key, and that requires having proper educators. It seems to me that the West is failing on that score, which presages a fall. Ray Dalio has a book about civilizational health. Bad education is a sign of illness.

"A hammer is supposed to help one drive in nails, not have an opinion about what kind of nails it likes."

Another construction metaphor? Sigh. Unfortunately for this argument, different hammers are in fact designed for hitting different things. That is why after thousands of years we haven't all converged on One Best Hammer. Just this week I've used two different kinds of hammers for two different kinds of nails.

"Some of us want to see that full diversity of views ... knowing multiple European languages is no longer enough, when it used to be."

Oh we got both kinds of viewpoints here: Romantic *and* Germanic!

@Ted: :s/what kind of nails it likes/what brand of nails it likes/g

@ Old Unix Geek,

Yeah you're probably right - it's better that there is no interference when information is provided to the public. Perhaps with context provided with the information. As you say, education is key.

It's just problematic as malign influencers will always game the system.

It turns out that being the conduit for 'all' the information is a bit of a poisoned chalice.

"that’s not a good sign for Bing"

Yes. I find Bing's results to be pretty consistently less useful than Google's. My only point is that much of this is probably a matter of them being a worse search engine (and having always been worse), rather than some kind of intentional or recent attempt to remove stuff from their index.

"Reality itself is neither liberal nor conservative"

Obviously. As you pointed out, it was a joke. I'm just saying that "search engines need to rank falsehoods higher than the truth, or else they are biased" is a hell of a take.

"Instead we cover our feet in leather"

Exactly. But your point is that people should wear shoes without soles, and just get thicker skin, because shoe manufacturers shouldn't be protecting people's feet.

Hi Michael,

I saw this after reading Jesse's article - I'm facing the same problem. In addition to writing a report of the situation, I've used the opportunity to check out the most popular search engines, see which one basically "copy" Bing, and which ones seem to be independent.

Incident report (still unresolved) and the search engine lists are here:
https://io.bikegremlin.com/28530/microsoft-bing-serp-gone-overnight/

I began looking into this in July. We use Duck Duck Go (Bing) for our internal search, but site: searches went from having thousands of pages to between 10 and 55. In other words, they were absolutely useless for our readers. The pages that Bing did have were really, really old (we’re talking early 2000s) for the most part, including some we had not linked since 2005. We had to switch to Mojeek, which had about 3,600 entries for us.

I’ll cut to the chase: Bing has tanked. The index appears to have plunged to fewer than 1,000 million pages. That’s smaller than Inktomi or Alltheweb back in the 2000s. I don’t know why, and I doubt they do.

Here was what I discovered:

https://jackyan.com/blog/2022/07/bing-has-tanked/

Try feeding in different site: searches into Google, Mojeek and Bing and you’ll see just how tiny Bing’s index is.

https://jackyan.com/blog/2022/08/mojeek-shows-more-in-its-search-results-than-google/

For this site, site:mjtsai.com returns the following:

Google: 9,220 claimed
Mojeek: 19,774
Bing: 78

Bing will even repeat results so you can’t believe what it says when it has found x results for your site.

https://jackyan.com/blog/2022/08/more-of-bings-follies-they-just-keep-coming/

[…] on links collected by Michael Tsai, this appears to be a somewhat common issue where Bing — and, consequently, the many small search […]