Consumer Civil War: Marketing As SIGINT/Information Warfare

I will have some pictures from Hevron and Kiryat Arba up next week. In the meantime, I will be using this blog to talk about something interesting that just occurred to me today, involving online marketing, signals intelligence, information warfare and the future.

In World War Two, the US-led coalition won in very large part because of American and British SIGINT. The Allies had cracked the German and Japanese codes and were able to see exactly what the Axis was doing and where (with some exceptions like the German Wacht Am Rhein offensive in the winter of '44-'45, but that was prepared for in total radio silence, using inner lines.) Now the details of the program, ULTRA/MAGIC, are widely available. This was what created the modern digital computer. Without this program, none of the other American efforts would have been very successful: the Manhattan Project, for instance, would not have been able to deliver a nuclear weapon to the Japanese Islands without Allied domination of the Pacific, the air base in Saipan and so forth. Without ULTRA/MAGIC, the Americans would not have been able to win the Battle of Midway, which was the tipping point of the Pacific Theater, and would not have been able to achieve those pre-requisites.



I am not sure how good Soviet strategic SIGINT was, but I suspect that it was alright; even if the Soviets hadn't developed their own code-breaking expertise to a comparable level by 1941, they were getting anything their hearts desired on the back channel between their representatives in the US and the highest echelons of the American government like Alger Hiss. If Lend Lease involved sending the USSR uranium over the Great Falls, MT air bridge, I would be surprised if there was no exchange of cryptological expertise. If not, the American government was deeply infiltrated by people with double loyalties, and so at the very least the intelligence products of ULTRA/MAGIC, sanitized to conceal the source, would have appeared on Stalin's desk in a timely manner.

Now, one of the reasons that the US had this incredible cryptological expertise was that it had spent the prior 20 years honing a small but very effective cryptological apparatus against both state and commercial targets. For instance, the father of American SIGINT, William Friedman, broke the Japanese PURPLE code before Pearl Harbor. He had been running the Signals Intelligence Service for the War Department since 1929. His wife, Elizabeth Friedman, had been instrumental in breaking the codes of rum runners, which eventually became quite sophisticated. Here I could talk about the instrumentality of high-IQ Jews to every aspect of America's 20th century global domination, mentioning the ones in the Manhattan Project, the father of the nuclear submarine fleet, Admiral Rickover, etc., but that would be a distraction.

The reason that this program took the shape it did was that the global communication system was highly constrained. On one hand, there were telegraph cables, on the other, long-range radio communication bouncing HF off the inonosphere. HF is quite constrained in bandwidth because of the Shannon Theorem, and the technology involved was pretty primitive, so global communication was largely done by governments and big corporations. Another interesting tangent, which I will not go into in any depth, is the extent of the cryptographic apparati of pre-war global corporations. Certainly, they had quite well-developed HUMINT apparati; the OSS, in its initial stages, came out of the latter. I suspect that they had very robust cryptography as well, and that given the close connections between USG and Wall Street in the 40 years leading up to WW2, were privy to the latest governmental SIGINT developments.

Anyway, once the Allies started reading Axis communications, they were faced with the problem of how to capitalize on this information, without revealing that they were capitalizing upon it. Obviously, they were helped in this endeavor by their enemies bureaucratic inertia, resistance to change, cognitive biases, etc. But still, if out of all the enemy's convoys you keep sinking the ones they most need, and do so when you really have no way of knowing where they are aside from by listening in on strategic channels, the enemy will wise up. Then all your work is for nothing. Cracking codes informs your actions, your actions bring you success but also give the enemy data. Analyzing this data reveals that you must have cracked the codes.

So, the Allies had to work very hard to create fake data and feed it to the enemy to obscure the source of their success. Neal Stephenson's Cryptonomicon addresses this at length, with a fictional joint Anglo-American commando unit whose entire purpose is to, e.g., "accidentally" discover convoys crossing the Mediterranean and send out an alert that the enemy can hear, so that there is a plausible cover for sinking those convoys. Operation Mincemeat involved planting fake American plans on a dead guy, then throwing the body out of a plane to wash up on shores controlled by Germans. And so on.

So not only were the combatants gathering information on each other in order to inform their own actions, but they were also using disinformation to influence each other's actions. As someone once told me, "every conversation is either an interrogation or an indoctrination." And of course every interaction is a conversation, in that data flows back and forth.

Today, bandwidth and processing power is plentiful. We all have more of each than the combatants of WW2 (though brains are in shorter supply.) So what happens online, in our conversation space?

Well, the internet consists of lots of companies competing for consumers' and each others' attention, information and (eventually) money. The consumers are also playing their own games, using the internet for similar purposes. Again, every internet interaction is either an indoctrination or an interrogation, or both at the same time, perhaps from both sides. This mess is called "marketing," and it is exactly like the information warfare of WW2, only with millions of independent participants.

Initially, internet marketing was pretty primitive, with things like banner ads, pop-up windows, Nigerian spam emails and so on, driving you to buy things or reveal private information. This stuff was disinformation, in other words. We all got pretty good at blocking that out very quickly. Now, there are even plugins for Google Chrome like AdBlocker which identify any code on a webpage that tells your browser to show an ad and stop it from displaying. So we started using automated interrogation and analysis to identify disinformation and ignore it.

Of course, just by going to a site, you generate data about yourself which the owners of the site gather and not only use for themselves but also sell to other companies. Data like, say, where you're from, your age, your gender, what site you visited when, and so on. This is exactly analogous to the interrogation part of WW2 SIGINT. That stuff can then be used to provide an analogue of disinformation; if I know that you are surfing my airline tickets site from a Mac, I can infer that you are tolerant of higher prices. I can then show you higher prices than I would have shown someone looking for the exact same flight from a Linux computer or a PC. Or, if I know you are a 25-45 year old head of a household in a flood-prone area, I can have my site show you a news feed widget on the side of the screen, which will just happen to spotlight the direst prediction of a hurricane coming your way that I can find from a mainstream source, and then have a banner showing you Home Depot storm supplies on the news site when you click on the hurricane prediction. If you're a 30 year old mother living in the same area, maybe I'll show you the same hurricane prediction, but the news story will have a special on diapers and formula in your area-better stock up!

From the perspective of the user, the problem is simple: the tactics described above are designed to make you spend money you wouldn't have spent otherwise. If you would have spent it anyway, there would have been no point in running a marketing campaign. That damn hurricane failed to materialize, there was no flood, but you went and spent two hundred bucks on diapers and two by fours anyway. You spent hundreds more dollars on that flight than you would have if you'd bought it from a Lenovo. And you're kind of outclassed. I mean, these guys have degrees from Stanford and decades of experience-you're just trying to read some stupid blog in your free time.

Of course, there is a reason you put up with it-you are getting something in return. It's still better than buying your tickets from an airline counter. And that hurricane COULD have hit your town. If there wasn't anything in it for you, you wouldn't be online, or at least be on the darknet. Just like the Allies, the companies have to put a lot of good, relevant, accurate data in there so that you a) show up to reveal information about yourself, b) imbibe the disinformation they feed you. Otherwise, no point to the whole exercise.

What happens is an arms race. For instance, before, people used to make major purchasing decisions by consulting with a salesman, whose name they would get from an ad or from a referral. He had the chance to estimate the customer and create the biggest possible margin (difference in cost to his company and price to the customer) that the latter was willing to pay. Now, they are doing extensive research online, going through multiple sources of information (interrogation), narrowing down a set of purchasing options, then contacting multiple sellers for a price quotes. They might even tell the seller a price quoted by his competition, true or false, to get him to drop his price, using disinformation! Now, instead of being able to sell you your two-by-fours at a markup because you panicked over a hurricane prediction I fed you in the first place, I am stuck: you scouted other news sites, realized the chances of the hurricane coming your way were low, then looked at my competition, found the cheapest two-by-fours and are demanding I match that price.

So marketers respond by creating "content marketing." In other words, writing largely accurate forum and blog posts, white papers, webinars, Twitter updates and so on, and either sneaking their message in there, or using them as bait to gather potential customers, or just framing the discourse to increase the awareness of and perceived need for their services. Since we tend to assume that multiple sources of information are independent, and independent sources of information agreeing on something tends to mean that whatever they agree on is true, a good disinformation tactic is to use sock puppets, multiple online personae being controlled by the same individual, speaking in different voices and framing the discourse so that one of the things they agree on is your intended disinformation. In democratic politics, this is known as astroturfing. Once you have enough sock puppets speaking with some authority, others will join in of their own free will-humans are, after all, pack animals. The Communists used to call these kinds of retransmitters fellow travelers.

Of course, for this to work, there has to be a relatively high signal-to-noise ratio in your data. Lots of indubitably true and useful stuff. The disinformation has to be so diluted as to be practically imperceptible If you realize that someone is using the stream you drink from as a toilet, you will find another stream to drink from. Thus, Google's Don't Be Evil policy: in English, that means, don't let the users realize you are making them drink your waste.

Unfortunately, online it is difficult to find a clean source of water. I think a more viable option is to sell filters. How do you identify the disinformation in a white paper that is full of good, true and useful information, assuming you are actually interested in the latter? One approach is to use network analysis to identify sock puppets and fellow travelers. This could then be corroborated by analyzing the data and finding the "key" of the actual marketer transmitting through his sock puppets. Once this is done, you can compare other sources which do not have this key and are not associated with fellow travelers, find information commonalities, and then label the differences as presumable disinformation.

Now I need to think about how to automate this process. What potential customers exist for vetting of open-source data? What is their willingness to pay?