Читать книгу The Code Book: The Secret History of Codes and Code-breaking - Simon Singh, Simon Singh - Страница 9

For centuries, the simple monoalphabetic substitution cipher had been sufficient to ensure secrecy. The subsequent development of frequency analysis, first in the Arab world and then in Europe, destroyed its security. The tragic execution of Mary Queen of Scots was a dramatic illustration of the weaknesses of monoalphabetic substitution, and in the battle between cryptographers and cryptanalysts it was clear that the cryptanalysts had gained the upper hand. Anybody sending an encrypted message had to accept that an expert enemy codebreaker might intercept and decipher their most precious secrets.

The onus was clearly on the cryptographers to concoct a new, stronger cipher, something that could outwit the cryptanalysts. Although this cipher would not emerge until the end of the sixteenth century, its origins can be traced back to the fifteenth-century Florentine polymath Leon Battista Alberti. Born in 1404, Alberti was one of the leading figures of the Renaissance – a painter, composer, poet and philosopher, as well as the author of the first scientific analysis of perspective, a treatise on the housefly and a funeral oration for his dog. He is probably best known as an architect, having designed Rome’s first Trevi Fountain and having written De re aedificatoria, the first printed book on architecture, which acted as a catalyst for the transition from Gothic to Renaissance design.

Sometime in the 1460s, Alberti was wandering through the gardens of the Vatican when he bumped into his friend Leonardo Dato, the pontifical secretary, who began chatting to him about some of the finer points of cryptography. This casual conversation prompted Alberti to write an essay on the subject, outlining what he believed to be a new form of cipher. At the time, all substitution ciphers required a single cipher alphabet for encrypting each message. However, Alberti proposed using two or more cipher alphabets, switching between them during encipherment, thereby confusing potential cryptanalysts.

Plain alphabet a b c d e f g h i j k l m n o p q r s t u v w x y z

Cipher alphabet 1 F Z B V K I X A Y M E P L S D H J O R G N Q C U T W

Cipher alphabet 2 G O X B F W T H Q I L A P Z J D E S V Y C R K U H N

For example, here we have two possible cipher alphabets, and we could encrypt a message by alternating between them. To encrypt the message hello, we would encrypt the first letter according to the first cipher alphabet, so that h becomes A, but we would encrypt the second letter according to the second cipher alphabet, so that e becomes F. To encrypt the third letter we return to the first cipher alphabet, and to encrypt the fourth letter we return to the second alphabet. This means that the first I is enciphered as P, but the second I is enciphered as A. The final letter, o, is enciphered according to the first cipher alphabet and becomes D. The complete ciphertext reads AFPAD. The crucial advantage of Alberti’s system is that the same letter in the plaintext does not necessarily appear as the same letter in the ciphertext, so the repeated I in hello is enciphered differently in each case. Similarly, the repeated A in the ciphertext represents a different plaintext letter in each case, first h and then I.

Although he had hit upon the most significant breakthrough in encryption for over a thousand years, Alberti failed to develop his concept into a fully formed system of encryption. That task fell to a diverse group of intellectuals, who built on his initial idea. First came Johannes Trithemius, a German abbot born in 1462, then Giovanni Porta, an Italian scientist born in 1535, and finally Blaise de Vigenère, a French diplomat born in 1523. Vigenère became acquainted with the writings of Alberti, Trithemius and Porta when, at the age of twenty-six, he was sent to Rome on a two-year diplomatic mission. To start with, his interest in cryptography was purely practical and was linked to his diplomatic work. Then, at the age of thirty-nine, Vigenère decided that he had accumulated enough money for him to be able to abandon his career and concentrate on a life of study. It was only then that he examined in detail the ideas of Alberti, Trithemius and Porta, weaving them into a coherent and powerful new cipher.

Figure 11 Blaise de Vigenère.

Cliché Bibliothèque Nationale de France, Paris, France.

Table 3 A Vigenère square.

Although Alberti, Trithemius and Porta all made vital contributions, the cipher is known as the Vigenère cipher in honour of the man who developed it into its final form. The strength of the Vigenère cipher lies in its using not one, but 26 distinct cipher alphabets to encrypt a message. The first step in encipherment is to draw up a so-called Vigenère square, as shown in Table 3, a plaintext alphabet followed by 26 cipher alphabets, each shifted by one letter with respect to the previous alphabet. Hence, row 1 represents a cipher alphabet with a Caesar shift of 1, which means that it could be used to implement a Caesar shift cipher in which every letter of the plaintext is replaced by the letter one place further on in the alphabet. Similarly, row 2 represents a cipher alphabet with a Caesar shift of 2, and so on. The top row of the square, in lower case, represents the plaintext letters. You could encipher each plaintext letter according to any one of the 26 cipher alphabets. For example, if cipher alphabet number 2 is used, then the letter a is enciphered as C, but if cipher alphabet number 12 is used, then a is enciphered as M.

If the sender were to use just one of the cipher alphabets to encipher an entire message, this would effectively be a simple Caesar cipher, which would be a very weak form of encryption, easily deciphered by an enemy interceptor. However, in the Vigenère cipher a different row of the Vigenère square (a different cipher alphabet) is used to encrypt different letters of the message. In other words, the sender might encrypt the first letter according to row 5, the second according to row 14, the third according to row 21, and so on.

To unscramble the message, the intended receiver needs to know which row of the Vigenère square has been used to encipher each letter, so there must be an agreed system of switching between rows. This is achieved by using a keyword. To illustrate how a keyword is used with the Vigenère square to encrypt a short message, let us encipher divert troops to east ridge, using the keyword WHITE. First of all, the keyword is spelt out above the message, and repeated over and over again so that each letter in the message is associated with a letter from the keyword. The ciphertext is then generated as follows. To encrypt the first letter, d, begin by identifying the key letter above it, W, which in turn defines a particular row in the Vigenère square. The row beginning with W, row 22, is the cipher alphabet that will be used to find the substitute letter for the plaintext d. We look to see where the column headed by d intersects the row beginning with W, which turns out to be at the letter Z. Consequently, the letter d in the plaintext is represented by Z in the ciphertext.

Keyword W H I T E W H I T E W H I T E W H I T E W H I

Plaintext d i v e r t t r o o p s t o e a s t r i d g e

Ciphertext Z P D X V P A Z H S L Z B H I W Z B K M Z N M

Table 4 A Vigenère square with the rows defined by the keyword WHITE highlighted. Encryption is achieved by switching between the five highlighted cipher alphabets, defined by W, H, I, T and E.

To encipher the second letter of the message, i, the process is repeated. The key letter above i is H, so it is encrypted via a different row in the Vigenère square: the H row (row 7) which is a new cipher alphabet. To encrypt i, we look to see where the column headed by i intersects the row beginning with H, which turns out to be at the letter P. Consequently, the letter i in the plaintext is represented by P in the ciphertext. Each letter of the keyword indicates a particular cipher alphabet within the Vigenère square, and because the keyword contains five letters, the sender encrypts the message by cycling through five rows of the Vigenère square. The fifth letter of the message is enciphered according to the fifth letter of the keyword, E, but to encipher the sixth letter of the message we have to return to the first letter of the keyword. A longer keyword, or perhaps a keyphrase, would bring more rows into the encryption process and increase the complexity of the cipher. Table 4 shows a Vigenère square, highlighting the five rows (i.e. the five cipher alphabets) defined by the keyword WHITE.

The great advantage of the Vigenère cipher is that it is impregnable to the frequency analysis described in Chapter 1. For example, a cryptanalyst applying frequency analysis to a piece of ciphertext would usually begin by identifying the most common letter in the ciphertext, which in this case is Z, and then assume that this represents the most common letter in English, e. In fact, the letter Z represents three different letters, d, r and s, but not e. This is clearly a problem for the cryptanalyst. The fact that a letter which appears several times in the ciphertext can represent a different plaintext letter on each occasion generates tremendous ambiguity for the cryptanalyst. Equally confusing is the fact that a letter which appears several times in the plaintext can be represented by different letters in the ciphertext. For example, the letter o is repeated in troops, but it is substituted by two different letters – the oo is enciphered as HS.

As well as being invulnerable to frequency analysis, the Vigenère cipher has an enormous number of keys. The sender and receiver can agree on any word in the dictionary, any combination of words, or even fabricate words. A cryptanalyst would be unable to crack the message by searching all possible keys because the number of options is simply too great.

Vigenère’s work culminated in his Traicté des Chiffres (‘A Treatise on Secret Writing’), published in 1586. Ironically, this was the same year that Thomas Phelippes was breaking the cipher of Mary Queen of Scots. If only Mary’s secretary had read this treatise, he would have known about the Vigenère cipher, Mary’s messages to Babington would have baffled Phelippes, and her life might have been spared.

Because of its strength and its guarantee of security, it would seem natural that the Vigenère cipher would be rapidly adopted by cipher secretaries around Europe. Surely they would be relieved to have access, once again, to a secure form of encryption? On the contrary, cipher secretaries seem to have spurned the Vigenère cipher. This apparently flawless system would remain largely neglected for the next two centuries.

From Shunning Vigenère to the Man in the Iron Mask

The traditional forms of substitution cipher, those that existed before the Vigenère cipher, were called monoalphabetic substitution ciphers because they used only one cipher alphabet per message. In contrast, the Vigenère cipher belongs to a class known as polyalphabetic, because it employs several cipher alphabets per message. The polyalphabetic nature of the Vigenère cipher is what gives it its strength, but it also makes it much more complicated to use. The additional effort required in order to implement the Vigenère cipher discouraged many people from employing it.

For many seventeenth-century purposes, the monoalphabetic substitution cipher was perfectly adequate. If you wanted to ensure that your servant was unable to read your private correspondence, or if you wanted to protect your diary from the prying eyes of your spouse, then the old-fashioned type of cipher was ideal. Monoalphabetic substitution was quick, easy to use, and secure against people unschooled in cryptanalysis. In fact, the simple monoalphabetic substitution cipher endured in various forms for many centuries (see Appendix D). For more serious applications, such as military and government communications, where security was paramount, the straightforward monoalphabetic cipher was clearly inadequate. Professional cryptographers in combat with professional cryptanalysts needed something better, yet they were still reluctant to adopt the polyalphabetic cipher because of its complexity. Military communications, in particular, required speed and simplicity, and a diplomatic office might be sending and receiving hundreds of messages each day, so time was of the essence. Consequently, cryptographers searched for an intermediate cipher, one that was harder to crack than a straightforward monoalphabetic cipher, but one that was simpler to implement than a polyalphabetic cipher.

The various candidates included the remarkably effective homophonic substitution cipher. Here, each letter is replaced with a variety of substitutes, the number of potential substitutes being proportional to the frequency of the letter. For example, the letter a accounts for roughly 8 per cent of all letters in written English, and so we would assign eight symbols to represent it. Each time a appears in the plaintext it would be replaced in the ciphertext by one of the eight symbols chosen at random, so that by the end of the encipherment each symbol would constitute roughly 1 per cent of the enciphered text. By comparison, the letter b accounts for only 2 per cent of all letters, and so we would assign only two symbols to represent it. Each time b appears in the plaintext either of the two symbols could be chosen, and by the end of the encipherment each symbol would also constitute roughly 1 per cent of the enciphered text. This process of allotting varying numbers of symbols to act as substitutes for each letter continues throughout the alphabet, until we get to z, which is so rare that it has only one symbol to act as a substitute. In the example given in Table 5, the substitutes in the cipher alphabet happen to be two-digit numbers, and there are between one and twelve substitutes for each letter in the plain alphabet, depending on each letter’s relative abundance.

Table 5 An example of a homophonic substitution cipher. The top row represents the plain alphabet, while the numbers below represent the cipher alphabet, with several options for frequently occurring letters.

We can think of all the two-digit numbers that correspond to the plaintext letter a as effectively representing the same sound in the ciphertext, namely the sound of the letter a. Hence the origin of the term homophonic substitution, homos meaning ‘same’ and phonos meaning ‘sound’ in Greek. The point of offering several substitution options for popular letters is to balance out the frequencies of symbols in the ciphertext. If we enciphered a message using the cipher alphabet in Table 5, then every number would constitute roughly 1 per cent of the entire text. If no symbol appears more frequently than any other, then this would appear to defy any potential attack via frequency analysis. Perfect security? Not quite.

The ciphertext still contains many subtle clues for the clever cryptanalyst. As we saw in Chapter 1, each letter in the English language has its own personality, defined according to its relationship with all the other letters, and these traits can still be discerned even if the encryption is by homophonic substitution. In English, the most extreme example of a letter with a distinct personality is the letter q, which is only followed by one letter, namely u. If we were attempting to decipher a ciphertext, we might begin by noting that q is a rare letter, and is therefore likely to be represented by just one symbol, and we know that u, which accounts for roughly 3 per cent of all letters, is probably represented by three symbols. So, if we find a symbol in the ciphertext that is only ever followed by three particular symbols, then it would be sensible to assume that the first symbol represents q and the other three symbols represent u. Other letters are harder to spot, but are also betrayed by their relationships to one another. Although the homophonic cipher is breakable, it is much more secure than a straightforward monoalphabetic cipher.

A homophonic cipher might seem similar to a polyalphabetic cipher inasmuch as each plaintext letter can be enciphered in many ways, but there is a crucial difference, and the homophonic cipher is in fact a type of monoalphabetic cipher. In the table of homophones shown above, the letter a can be represented by eight numbers. Significantly, these eight numbers represent only the letter a. In other words, a plaintext letter can be represented by several symbols, but each symbol can only represent one letter. In a polyalphabetic cipher, a plaintext letter will also be represented by different symbols, but, even more confusingly, these symbols will represent different letters during the course of an encipherment.

Perhaps the fundamental reason why the homophonic cipher is considered monoalphabetic is that once the cipher alphabet has been established, it remains constant throughout the process of encryption. The fact that the cipher alphabet contains several options for encrypting each letter is irrelevant. However, a cryptographer who is using a polyalphabetic cipher must continually switch between distinctly different cipher alphabets during the process of encryption.

By tweaking the basic monoalphabetic cipher in various ways, such as adding homophones, it became possible to encrypt messages securely, without having to resort to the complexities of the polyalphabetic cipher. One of the strongest examples of an enhanced monoalphabetic cipher was the Great Cipher of Louis XIV. The Great Cipher was used to encrypt the king’s most secret messages, protecting details of his plans, plots and political schemings. One of these messages mentioned one of the most enigmatic characters in French history, the Man in the Iron Mask, but the strength of the Great Cipher meant that the message and its remarkable contents would remain undeciphered and unread for two centuries.

The Great Cipher was invented by the father-and-son team of Antoine and Bonaventure Rossignol. Antoine had first come to prominence in 1626 when he was given a coded letter captured from a messenger leaving the besieged city of Réalmont. Before the end of the day he had deciphered the letter, revealing that the Huguenot army which held the city was on the verge of collapse. The French, who had previously been unaware of the Huguenots’ desperate plight, returned the letter accompanied by a decipherment. The Huguenots, who now knew that their enemy would not back down, promptly surrendered. The decipherment had resulted in a painless French victory.

The power of codebreaking became obvious, and the Rossignols were appointed to senior positions in the court. After serving Louis XIII, they then acted as cryptanalysts for Louis XIV, who was so impressed that he moved their offices next to his own apartments so that Rossignol père et fils could play a central role in shaping French diplomatic policy. One of the greatest tributes to their abilities is that the word rossignol became French slang for a device that picks locks, a reflection of their ability to unlock ciphers.

The Rossignols’ prowess at cracking ciphers gave them an insight into how to create a stronger form of encryption, and they invented the so-called Great Cipher. The Great Cipher was so secure that it defied the efforts of all enemy cryptanalysts attempting to steal French secrets. Unfortunately, after the death of both father and son, the Great Cipher fell into disuse and its exact details were rapidly lost, which meant that enciphered papers in the French archives could no longer be read. The Great Cipher was so strong that it even defied the efforts of subsequent generations of codebreakers.

Historians knew that the papers encrypted by the Great Cipher would offer a unique insight into the intrigues of seventeenth-century France, but even by the end of the nineteenth century they were still unable to decipher them. Then, in 1890, Victor Gendron, a military historian researching the campaigns of Louis XIV, unearthed a new series of letters enciphered with the Great Cipher. Unable to make sense of them, he passed them on to Commandant Étienne Bazeries, a distinguished expert in the French Army’s Cryptographic Department. Bazeries viewed the letters as the ultimate challenge, and he spent the next three years of his life attempting to decipher them.

The encrypted pages contained thousands of numbers, but only 587 different ones. It was clear that the Great Cipher was more complicated than a straightforward substitution cipher, because this would require just 26 different numbers, one for each letter. Initially, Bazeries thought that the surplus of numbers represented homophones, and that several numbers represented the same letter. Exploring this avenue took months of painstaking effort, all to no avail. The Great Cipher was not a homophonic cipher.

Next, he hit upon the idea that each number might represent a pair of letters, or a digraph. There are only 26 individual letters, but there are 676 possible pairs of letters, and this is roughly equal to the variety of numbers in the ciphertexts. Bazeries attempted a decipherment by looking for the most frequent numbers in the ciphertexts (22, 42, 124, 125 and 341), assuming that these probably stood for the commonest French digraphs (es, en, ou, de, nt). In effect, he was applying frequency analysis at the level of pairs of letters. Unfortunately, again after months of work, this theory also failed to yield any meaningful decipherments.

Bazeries must have been on the point of abandoning his obsession, when a new line of attack occurred to him. Perhaps the digraph idea was not so far from the truth. He began to consider the possibility that each number represented not a pair of letters, but rather a whole syllable. He attempted to match each number to a syllable, the most frequently occurring numbers presumably representing the commonest French syllables. He tried various tentative permutations, but they all resulted in gibberish – until he succeeded in identifying one particular word. A cluster of numbers (124-22-125-46-345) appeared several times on each page, and Bazeries postulated that they represented les-en-ne-mi-s, that is, ‘les ennemis’. This proved to be a crucial breakthrough.

Bazeries was then able to continue by examining other parts of the ciphertexts where these numbers appeared within different words. He then inserted the syllabic values derived from ‘les enemis’, which revealed parts of other words. As crossword addicts know, when a word is partly completed it is often possible to guess the remainder of the word. As Bazeries completed new words, he also identified further syllables, which in turn led to other words, and so on. Frequently he would be stumped, partly because the syllabic values were never obvious, partly because some of the numbers represented single letters rather than syllables, and partly because the Rossignols had laid traps within the cipher. For example, one number represented neither a syllable nor a letter, but instead deviously deleted the previous number.

When the decipherment was eventually completed, Bazeries became the first person for two hundred years to witness the secrets of Louis XIV. The newly deciphered material fascinated historians, who focused on one tantalising letter in particular. It seemed to solve one of the great mysteries of the seventeenth century: the true identity of the Man in the Iron Mask.

The Man in the Iron Mask has been the subject of much speculation ever since he was first imprisoned at the French fortress of Pignerole in Savoy. When he was transferred to the Bastille in 1698, peasants tried to catch a glimpse of him, and variously reported him as being short or tall, fair or dark, young or old. Some even claimed that he was a she. With so few facts, everyone from Voltaire to Benjamin Franklin concocted their own theory to explain the case of the Man in the Iron Mask. The most popular conspiracy theory relating to the Mask (as he is sometimes called) suggests that he was the twin of Louis XIV, condemned to imprisonment in order to avoid any controversy over who was the rightful heir to the throne. One version of this theory argues that there existed descendants of the Mask and an associated hidden royal bloodline. A pamphlet published in 1801 said that Napoleon himself was a descendant of the Mask, a rumour which, since it enhanced his position, the emperor did not deny.

The myth of the Mask even inspired poetry, prose and drama. In 1848 Victor Hugo had begun writing a play entitled Twins, but when he found that Alexandre Dumas had already plumped for the same plot, he abandoned the two acts he had written. Ever since, it has been Dumas’s name that we associate with the story of the Man in the Iron Mask. The success of his novel reinforced the idea that the Mask was related to the king, and this theory has persisted despite the evidence revealed in one of Bazeries’s decipherments.

Bazeries had deciphered a letter written by François de Louvois, Louis XIV’s Minister of War, which began by recounting the crimes of Vivien de Bulonde, the commander responsible for leading an attack on the town of Cuneo, on the French-Italian border. Although he was ordered to stand his ground, Bulonde became concerned about the arrival of enemy troops from Austria and fled, leaving behind his munitions and abandoning many of his wounded soldiers. According to the Minister of War, these actions jeopardised the whole Piedmont campaign, and the letter made it clear that the king viewed Bulonde’s actions as an act of extreme cowardice:

His Majesty knows better than any other person the consequences of this act, and he is also aware of how deeply our failure to take the place will prejudice our cause, a failure which must be repaired during the winter. His Majesty desires that you immediately arrest General Bulonde and cause him to be conducted to the fortress of Pignerole, where he will be locked in a cell under guard at night, and permitted to walk the battlements during the day with a mask.

This was an explicit reference to a masked prisoner at Pignerole, and a sufficiently serious crime, with dates that seem to fit the myth of the Man in the Iron Mask. Does this solve the mystery? Not surprisingly, those favouring more conspiratorial solutions have found flaws in Bulonde as a candidate. For example, there is the argument that if Louis XIV was actually attempting to secretly imprison his unacknowledged twin, then he would have left a series of false trails. Perhaps the encrypted letter was meant to be deciphered. Perhaps the nineteenth-century codebreaker Bazeries had fallen into a seventeenth-century trap.

The Black Chambers

Reinforcing the monoalphabetic cipher by applying it to syllables or adding homophones might have been sufficient during the 1600s, but by the 1700s cryptanalysis was becoming industrialised, with teams of government cryptanalysts working together to crack many of the most complex monoalphabetic ciphers. Each European power had its own so-called Black Chamber, a nerve centre for deciphering messages and gathering intelligence. The most celebrated, disciplined and efficient Black Chamber was the Geheime Kabinets-Kanzlei in Vienna.

It operated according to a rigorous timetable, because it was vital that its nefarious activities should not interrupt the smooth running of the postal service. Letters which were supposed to be delivered to embassies in Vienna were first routed via the Black Chamber, arriving at 7 a.m. Secretaries melted seals, and a team of stenographers worked in parallel to make copies of the letters. If necessary, a language specialist would take responsibility for duplicating unusual scripts. Within three hours the letters had been resealed in their envelopes and returned to the central post office, so that they could be delivered to their intended destination. Mail merely in transit through Austria would arrive at the Black Chamber at 10 a.m., and mail leaving Viennese embassies for destinations outside Austria would arrive at 4 p.m. All these letters would also be copied before being allowed to continue on their journey. Each day a hundred letters would filter through the Viennese Black Chamber.

The copies were passed to the cryptanalysts, who sat in little kiosks, ready to tease out the meanings of the messages. As well as supplying the emperors of Austria with invaluable intelligence, the Viennese Black Chamber sold the information it harvested to other powers in Europe. In 1774 an arrangement was made with Abbot Georgel, the secretary at the French Embassy, which gave him access to a twice-weekly package of information in exchange for 1,000 ducats. He then sent these letters, which contained the supposedly secret plans of various monarchs, straight to Louis XV in Paris.

The Black Chambers were effectively making all forms of monoalphabetic cipher insecure. Confronted with such professional cryptanalytic opposition, cryptographers were at last forced to adopt the more complex but more secure Vigenère cipher. Gradually, cipher secretaries began to switch to using polyalphabetic ciphers. In addition to more effective cryptanalysis, there was another pressure that was encouraging the move towards securer forms of encryption: the development of the telegraph, and the need to protect telegrams from interception and decipherment.

Although the telegraph, together with the ensuing telecommunications revolution, came in the nineteenth century, its origins can be traced all the way back to 1753. An anonymous letter in a Scottish magazine described how a message could be sent across large distances by connecting the sender and receiver with 26 cables, one for each letter of the alphabet. The sender could then spell out the message by sending pulses of electricity along each wire. For example, to spell out hello, the sender would begin by sending a signal down the h wire, then down the e wire, and so on. The receiver would somehow sense the electrical current emerging from each wire and read the message. However, this ‘expeditious method of conveying intelligence’, as the inventor called it, was never constructed, because there were several technical obstacles that had to be overcome.

For example, engineers needed a sufficiently sensitive system for detecting electrical signals. In England, Sir Charles Wheatstone and William Fothergill Cooke built detectors from magnetised needles, which would be deflected in the presence of an incoming electric current. By 1839, the Wheatstone-Cooke system was being used to send messages between railway stations in West Drayton and Paddington, a distance of 29 km. The reputation of the telegraph and its remarkable speed of communication soon spread, and nothing did more to popularise its power than the birth of Queen Victoria’s second son, Prince Alfred, at Windsor on 6 August 1844. News of the birth was telegraphed to London, and within the hour The Times was on the streets announcing the news. It credited the technology that had enabled this feat, mentioning that it was ‘indebted to the extraordinary power of the Electro-Magnetic Telegraph’. The following year, the telegraph gained further fame when it helped capture John Tawell, who had murdered his mistress in Slough, and who had attempted to escape by jumping on to a London-bound train. The local police telegraphed Tawell’s description to London, and he was arrested as soon as he arrived at Paddington.

Meanwhile, in America, Samuel Morse had just built his first telegraph line, a system spanning the 60 km between Baltimore and Washington. Morse used an electromagnet to enhance the signal, so that upon arriving at the receiver’s end it was strong enough to make a series of short and long marks, dots and dashes, on a piece of paper. He also developed the now familiar Morse code for translating each letter of the alphabet into a series of dots and dashes, as given in Table 6. To complete his system he designed a sounder, so that the receiver would hear each letter as a series of audible dots and dashes.

Back in Europe, Morse’s approach gradually overtook the Wheatstone-Cooke system in popularity, and in 1851 a European form of Morse Code, which included accented letters, was adopted throughout the Continent. As each year passed, Morse code and the telegraph had an increasing influence on the world, enabling the police to capture more criminals, helping newspapers to bring the very latest news, providing valuable information for businesses, and allowing distant companies to make instantaneous deals.

However, guarding these often sensitive communications was a major concern. The Morse code itself is not a form of cryptography, because there is no concealment of the message. The dots and dashes are merely a convenient way to represent letters for the telegraphic medium; Morse code is effectively nothing more than an alternative alphabet. The problem of security arose primarily because anyone wanting to send a message would have to deliver it to a Morse code operator, who would then have to read it in order to transmit it. The telegraph operators had access to every message, and hence there was a risk that one company might bribe an operator in order to gain access to a rival’s communications. This problem was outlined in an article on telegraphy published in 1853 in England’s Quarterly Review:

Means should also be taken to obviate one great objection, at present felt with respect to sending private communications by telegraph – the violation of all secrecy – for in any case half-a-dozen people must be cognisant of every word addressed by one person to another. The clerks of the English Telegraph Company are sworn to secrecy, but we often write things that it would be intolerable to see strangers read before our eyes. This is a grievous fault in the telegraph, and it must be remedied by some means or other.

Table 6 International Morse Code symbols.

The solution was to encipher a message before handing it to the telegraph operator. The operator would then turn the ciphertext into Morse code before transmitting it. As well as preventing the operators from seeing sensitive material, encryption also stymied the efforts of any spy who might be tapping the telegraph wire. The polyalphabetic Vigenère cipher was clearly the best way to ensure secrecy for important business communications. It was considered unbreakable, and became known as le chiffre indéchiffrable. Cryptographers had, for the time being at least, a clear lead over the cryptanalysts.

Mr Babbage Versus the Vigenère Cipher

The most intriguing figure in nineteenth-century cryptanalysis is Charles Babbage, the eccentric British genius best known for developing the blueprint for the modern computer. He was born in 1791, the son of Benjamin Babbage, a wealthy London banker. When Charles married without his father’s permission, he no longer had access to the Babbage fortune, but he still had enough money to be financially secure, and he pursued the life of a roving scholar, applying his mind to whatever problem tickled his fancy. His inventions include the speedometer and the cowcatcher, a device that could be fixed to the front of steam locomotives to clear cattle from railway tracks. In terms of scientific breakthroughs, he was the first to realise that the width of a tree ring depended on that year’s weather, and he deduced that it was possible to determine past climates by studying ancient trees. He was also intrigued by statistics, and as a diversion he drew up a set of mortality tables, a basic tool for today’s insurance industry.

Babbage did not restrict himself to tackling scientific and engineering problems. The cost of sending a letter used to depend on the distance the letter had to travel, but Babbage pointed out that the cost of the labour required to calculate the price for each letter was more than the cost of the postage. Instead, he proposed the system we still use today – a single price for all letters, regardless of where in the country the addressee lives. He was also interested in politics and social issues, and towards the end of his life he began a campaign to get rid of the organ-grinders and street musicians who roamed London. He complained that the music ‘not infrequently gives rise to a dance by little ragged urchins, and sometimes half-intoxicated men, who occasionally accompany the noise with their own discordant voices. Another class who are great supporters of street music consists of ladies of elastic virtue and cosmopolitan tendencies, to whom it affords a decent excuse for displaying their fascinations at their open windows.’ Unfortunately for Babbage, the musicians fought back by gathering in large groups around his house and playing as loud as possible.

The turning point in Babbage’s scientific career came in 1821, when he and the astronomer John Herschel were examining a set of mathematical tables, the sort used as the basis for astronomical, engineering and navigational calculations. The two men were disgusted by the number of errors in the tables, which in turn would generate flaws in important calculations. One set of tables, the Nautical Ephemeris for Finding Latitude and Longitude at Sea, contained over a thousand errors. Indeed, many shipwrecks and engineering disasters were blamed on faulty tables.

These mathematical tables were calculated by hand, and the mistakes were simply the result of human error. This caused Babbage to exclaim, ‘I wish to God these calculations had been executed by steam!’ This marked the beginning of an extraordinary endeavour to build a machine capable of faultlessly calculating the tables to a high degree of accuracy. In 1823 Babbage designed ‘Difference Engine No. 1’, a magnificent calculator consisting of 25,000 precision parts, to be built with government funding. Although Babbage was a brilliant innovator, he was not a great implementer. After ten years of toil, he abandoned ‘Difference Engine No. 1’, cooked up an entirely new design, and set to work building ‘Difference Engine No. 2’.

When Babbage abandoned his first machine, the government lost confidence in him and decided to cut its losses by withdrawing from the project – it had already spent £17,470, enough to build a pair of battleships. It was probably this withdrawal of support that later prompted Babbage to make the following complaint: ‘Propose to an Englishman any principle, or any instrument, however admirable, and you will observe that the whole effort of the English mind is directed to find a difficulty, a defect, or an impossibility in it. If you speak to him of a machine for peeling a potato, he will pronounce it impossible: if you peel a potato with it before his eyes, he will declare it useless, because it will not slice a pineapple.’

Figure 12 Charles Babbage.

Science and Society Picture Library, London.

Lack of government funding meant that Babbage never completed Difference Engine No. 2. The scientific tragedy was that Babbage’s machine would have been a stepping stone to the Analytical Engine. Rather than merely calculating a specific set of tables, the Analytical Engine would have been able to solve a variety of mathematical problems depending on the instructions that it was given. In fact, the Analytical Engine provided the template for modern computers. The design included a ‘store’ (memory) and a ‘mill’ (processor), which would allow it to make decisions and repeat instructions, which are equivalent to the ‘IF … THEN … ’ and ‘LOOP’ commands in modern programming.

A century later, during the course of the Second World War, the first electronic incarnations of Babbage’s machine would have a profound effect on cryptanalysis, but, in his own lifetime, Babbage made an equally important contribution to codebreaking: he succeeded in breaking the Vigenère cipher, and in so doing he made the greatest breakthrough in cryptanalysis since the Arab scholars of the ninth century broke the monoalphabetic cipher by inventing frequency analysis. Babbage’s work required no mechanical calculations or complex computations. Instead, he employed nothing more than sheer cunning.

Babbage had become interested in ciphers at a very young age. In later life, he recalled how his childhood hobby occasionally got him into trouble: ‘The bigger boys made ciphers, but if I got hold of a few words, I usually found out the key. The consequence of this ingenuity was occasionally painful: the owners of the detected ciphers sometimes thrashed me, though the fault lay in their own stupidity.’ These beatings did not discourage him, and he continued to be enchanted by cryptanalysis. He wrote in his autobiography that ‘deciphering is, in my opinion, one of the most fascinating of arts’.

He soon gained a reputation within London society as a cryptanalyst prepared to tackle any encrypted message, and strangers would approach him with all sorts of problems. For example, Babbage helped a desperate biographer attempting to decipher the shorthand notes of John Flamsteed, England’s first Astronomer Royal. He also came to the rescue of a historian, solving a cipher of Henrietta Maria, wife of Charles I. In 1854, he collaborated with a barrister and used cryptanalysis to reveal crucial evidence in a legal case. Over the years, he accumulated a thick file of encrypted messages, which he planned to use as the basis for an authoritative book on cryptanalysis, entitled The Philosophy of Decyphering. The book would contain two examples of every kind of cipher, one that would be broken as a demonstration and one that would be left as an exercise for the reader. Unfortunately, as with many other of his grand plans, the book was never completed.

While most cryptanalysts had given up all hope of ever breaking the Vigenère cipher, Babbage was inspired to attempt a decipherment by an exchange of letters with John Hall Brock Thwaites, a dentist from Bristol with a rather innocent view of ciphers. In 1854, Thwaites claimed to have invented a new cipher, which, in fact, was equivalent to the Vigenère cipher. He wrote to the Journal of the Society of Arts with the intention of patenting his idea, apparently unaware that he was several centuries too late. Babbage wrote to the Society, pointing out that ‘the cypher … is a very old one, and to be found in most books’. Thwaites was unapologetic and challenged Babbage to break his cipher. Whether or not it was breakable was irrelevant to whether or not it was new, but Babbage’s curiosity was sufficiently aroused for him to embark on a search for a weakness in the Vigenère cipher.

Cracking a difficult cipher is akin to climbing a sheer cliff face. The cryptanalyst is seeking any nook or cranny which could provide the slightest purchase. In a monoalphabetic cipher the cryptanalyst will latch on to the frequency of the letters, because the commonest letters, such as e, t and a, will stand out no matter how they have been disguised. In the polyalphabetic Vigenère cipher the frequencies are much more balanced, because the keyword is used to switch between cipher alphabets. Hence, at first sight, the rock face seems perfectly smooth.

Remember, the great strength of the Vigenère cipher is that the same letter will be enciphered in different ways. For example, if the keyword is KING, then every letter in the plaintext can potentially be enciphered in four different ways, because the keyword contains four letters. Each letter of the keyword defines a different cipher alphabet in the Vigenère square, as shown in Table 7. The e column of the square has been highlighted to show how it is enciphered differently, depending on which letter of the keyword is defining the encipherment:

If the K of KING is used to encipher e, then the resulting ciphertext letter is O.

If the I of KING is used to encipher e, then the resulting ciphertext letter is M.

If the N of KING is used to encipher e, then the resulting ciphertext letter is R.

If the G of KING is used to encipher e, then the resulting ciphertext letter is K.

Table 7 A Vigenère square used in combination with the keyword KING. The keyword defines four separate cipher alphabets, so that the letter e may be encrypted as O, M, R or K.

Similarly, whole words will be enciphered in different ways: the word the, for example, could be enciphered as DPR, BUK, GNO or ZRM, depending on its position relative to the keyword. Although this makes cryptanalysis difficult, it is not impossible. The important point to note is that if there are only four ways to encipher the word the, and the original message contains several instances of the word the, then it is highly likely that some of the four possible encipherments will be repeated in the ciphertext. This is demonstrated in the following example, in which the line The Sun and the Man in the Moon has been enciphered using the Vigenère cipher and the keyword KING.

Keyword K I N G K I N G K I N G K I N G K I N G K I N G

Plaintext t h e s u n a n d t h e m a n i n t h e m o o n

Ciphertext D P R Y E V N T N B U K W I A O X B U K W W B T

The word the is enciphered as DPR in the first instance, and then as BUK on the second and third occasions. The reason for the repetition of BUK is that the second the is displaced by eight letters with respect to the third the, and eight is a multiple of the length of the keyword, which is four letters long. In other words, the second the was enciphered according to its relationship to the key word (the is directly below ING), and by the time we reach the third the, the keyword has cycled round exactly twice, to repeat the relationship, and hence repeat the encipherment.

Babbage realised that this sort of repetition provided him with exactly the foothold he needed in order to conquer the Vigenère cipher. He was able to define a series of relatively simple steps which could be followed by any cryptanalyst to crack the hitherto chiffre indéchiffrable. To demonstrate his brilliant technique, let us imagine that we have intercepted the ciphertext shown in Figure 13. We know that it was enciphered using the Vigenère cipher, but we know nothing about the original message, and the keyword is a mystery.

The first stage in Babbage’s cryptanalysis is to look for sequences of letters that appear more than once in the ciphertext. There are two ways that such repetitions could arise. The most likely is that the same sequence of letters in the plaintext has been enciphered using the same part of the key. Alternatively, there is a slight possibility that two different sequences of letters in the plaintext have been enciphered using different parts of the key, coincidentally leading to the identical sequence in the ciphertext. If we restrict ourselves to long sequences, then we largely discount the second possibility, and, in this case, we shall consider repeated sequences only if they are of four letters or more. Table 8 is a log of such repetitions, along with the spacing between the repetition. For example, the sequence E-F-I-Q appears in the first line of the ciphertext and then in the fifth line, shifted forward by 95 letters.

Figure 13 The ciphertext, enciphered using the Vigenère cipher.

As well as being used to encipher the plaintext into ciphertext, the keyword is also used by the receiver to decipher the ciphertext back into plaintext. Hence, if we could identify the keyword, deciphering the text would be easy. At this stage we do not have enough information to work out the keyword, but Table 8 does provide some very good clues as to its length. Having listed which sequences repeat themselves and the spacing between these repetitions, the rest of the table is given over to identifying the factors of the spacing – the numbers that will divide into the spacing. For example, the sequence W-C-X-Y-M repeats itself after 20 letters, and the numbers 1, 2, 4, 5, 10 and 20 are factors, because they divide perfectly into 20 without leaving a remainder. These factors suggest six possibilities:

(1) The key is 1 letter long and is recycled 20 times between encryptions.

(2) The key is 2 letters long and is recycled 10 times between encryptions.

(3) The key is 4 letters long and is recycled 5 times between encryptions.

(4) The key is 5 letters long and is recycled 4 times between encryptions.

(5) The key is 10 letters long and is recycled 2 times between encryptions.

(6) The key is 20 letters long and is recycled 1 time between encryptions.

The first possibility can be excluded, because a key that is only 1 letter long gives rise to a monoalphabetic cipher – only one row of the Vigenère square would be used for the entire encryption, and the cipher alphabet would remain unchanged; it is unlikely that a cryptographer would do this. To indicate each of the other possibilities, a is placed in the appropriate column of Table 8. Each indicates a potential key length.

Table 8 Repetitions and spacings in the ciphertext.

To identify whether the key is 2, 4, 5, 10 or 20 letters long, we need to look at the factors of all the other spacings. Because the keyword seems to be 20 letters or smaller, Table 8 lists those factors that are 20 or smaller for each of the other spacings. There is a clear propensity for a spacing divisible by 5. In fact, every spacing is divisible by 5. The first repeated sequence, E-F-I-Q, can be explained by a keyword of length 5 recycled nineteen times between the first and second encryptions. The second repeated sequence, P-S-D-L-P, can be explained by a keyword of length 5 recycled just once between the first and second encryptions. The third repeated sequence, W-C-X-Y-M, can be explained by a keyword of length 5 recycled four times between the first and second encryptions. The fourth repeated sequence, E-T-R-L, can be explained by a keyword of length 5 recycled twenty-four times between the first and second encryptions. In short, everything is consistent with a five-letter keyword.

Assuming that the keyword is indeed 5 letters long, the next step is to work out the actual letters of the keyword. For the time being, let us call the keyword L₁-L₂-L₃-L₄-L₅, such that L₁ represents the first letter of the keyword, and so on. The process of encipherment would have begun with enciphering the first letter of the plaintext according to the first letter of the keyword, L₁. The letter L₁ defines one row of the Vigenère square, and effectively provides a monoalphabetic substitution cipher alphabet for the first letter of the plaintext. However, when it comes to encrypting the second letter of the plaintext, the cryptographer would have used L₂ to define a different row of the Vigenère square, effectively providing a different monoalphabetic substitution cipher alphabet. The third letter of plaintext would be encrypted according to L₃, the fourth according to L₄, and the fifth according to L₅. Each letter of the keyword is providing a different cipher alphabet for encryption. However, the sixth letter of the plaintext would once again be encrypted according to L₁, the seventh letter of the plaintext would once again be encrypted according to L₂, and the cycle repeats itself thereafter. In other words, the polyalphabetic cipher consists of five monoalphabetic ciphers, each monoalphabetic cipher is responsible for encrypting one-fifth of the entire message, and, most importantly, we already know how to cryptanalyse monoalphabetic ciphers.

We proceed as follows. We know that one of the rows of the Vigenère square, defined by L₁, provided the cipher alphabet to encrypt the 1st, 6th, 11th, 16th, … letters of the message. Hence, if we look at the 1st, 6th, 11th, 16th, … letters of the ciphertext, we should be able to use old-fashioned frequency analysis to work out the cipher alphabet in question. Figure 14 shows the frequency distribution of the letters that appear in the 1st, 6th, 11th, 16th, … positions of the ciphertext, which are W, I, R, E, …. At this point, remember that each cipher alphabet in the Vigenère square is simply a standard alphabet shifted by a value between 1 and 26. Hence, the frequency distribution in Figure 14 should have similar features to the frequency distribution of a standard alphabet, except that it will have been shifted by some distance. By comparing the L₁ distribution with the standard distribution, it should be possible to work out the shift. Figure 15 shows the standard frequency distribution for a piece of English plaintext.

Figure 14 Frequency distribution for letters in the ciphertext encrypted using the L₁ cipher alphabet (number of occurrences).

Figure 15 Standard frequency distribution (number of occurrences based on a piece of plaintext containing the same number of letters as in the ciphertext).

The standard distribution has peaks, plateaus and valleys, and to match it with the L₁ cipher distribution we look for the most outstanding combination of features. For example, the three spikes at R-S-T in the standard distribution (Figure 15) and the long depression to its right that stretches across six letters from U to Z together form a very distinctive pair of features. The only similar features in the L₁ distribution (Figure 14) are the three spikes at V-W-X, followed by the depression stretching six letters from Y to D. This would suggest that all the letters encrypted according to L₁ have been shifted four places, or that L₁ defines a cipher alphabet which begins E, F, G, H, …. In turn, this means that the first letter of the keyword, L1, is probably E. This hypothesis can be tested by shifting the L₁ distribution back four letters and comparing it with the standard distribution. Figure 16 shows both distributions for comparison. The match between the major peaks is very strong, implying that it is safe to assume that the keyword does indeed begin with E.

Figure 16 The L1 distribution shifted back four letters (top), compared with the standard frequency distribution (bottom). All major peaks and troughs match.

To summarise, searching for repetitions in the ciphertext has allowed us to identify the length of the keyword, which turned out to be five letters long. This allowed us to split the ciphertext into five parts, each one enciphered according to a monoalphabetic substitution as defined by one letter of the keyword. By analysing the fraction of the ciphertext that was enciphered according to the first letter of the keyword, we have been able to show that this letter, L₁, is probably E. This process is repeated in order to identify the second letter of the keyword. A frequency distribution is established for the 2nd, 7th, 12th, 17th, … letters in the ciphertext. Again, the resulting distribution, shown in Figure 17, is compared with the standard distribution in order to deduce the shift.

Figure 17 Frequency distribution for letters in the ciphertext encrypted using the L2 cipher alphabet (number of occurrences).

Figure 18 The L2 distribution shifted back twelve letters (top), compared with the standard frequency distribution (bottom). Most major peaks and troughs match.

This distribution is harder to analyse. There are no obvious candidates for the three neighbouring peaks that correspond to R-S-T. However, the depression that stretches from G to L is very distinct, and probably corresponds to the depression we expect to see stretching from U to Z in the standard distribution. If this were the case, we would expect the three R-S-T peaks to appear at D, E and F, but the peak at E is missing. For the time being, we shall dismiss the missing peak as a statistical glitch, and go with our initial reaction, which is that the depression from G to L is a recognisably shifted feature. This would suggest that all the letters encrypted according to L2 have been shifted twelve places, or that L₂ defines a cipher alphabet which begins M, N, O, P, … and that the second letter of the keyword, L2, is M. Once again, this hypothesis can be tested by shifting the L₂ distribution back twelve letters and comparing it with the standard distribution. Figure 18 shows both distributions, and the match between the major peaks is very strong, implying that it is safe to assume that the second letter of the keyword is indeed M.

I shall not continue the analysis; suffice to say that analysing the 3rd, 8th, 13th, … letters implies that the third letter of the keyword is I, analysing the 4th, 9th, 14th, … letters implies that the fourth letter is L, and analysing the 5th, 10th, 15th, … letters implies that the fifth letter is Y. The keyword is EMILY. It is now possible to reverse the Vigenère cipher and complete the cryptanalysis. The first letter of the ciphertext is W, and it was encrypted according to the first letter of the keyword, E. Working backwards, we look at the Vigenère square, and find W in the row beginning with E, and then we find which letter is at the top of that column. The letter is s, which must make it the first letter of the plaintext. By repeating this process, we see that the plaintext begins sittheedownandhavenoshamecheekbyjowl …. By inserting suitable word-breaks and punctuation, we eventually get:

Sit thee down, and have no shame,

Cheek by jowl, and knee by knee:

What care I for any name?

What for order or degree?

Let me screw thee up a peg:

Let me loose thy tongue with wine:

Callest thou that thing a leg?

Which is thinnest? thine or mine?

Thou shalt not be saved by works:

Thou hast been a sinner too:

Ruined trunks on withered forks,

Empty scarecrows, I and you!

Fill the cup, and fill the can:

Have a rouse before the mom:

Every moment dies a man,

Every moment one is born.

These are verses from a poem by Alfred Tennyson entitled ‘The Vision of Sin’. The keyword happens to be the first name of Tennyson’s wife, Emily Sellwood. I chose to use a section from this particular poem as an example for cryptanalysis because it inspired some curious correspondence between Babbage and the great poet. Being a keen statistician and compiler of mortality tables, Babbage was irritated by the lines ‘Every moment dies a man, Every moment one is born’, which are the last lines of the plaintext above. Consequently, he offered a correction to Tennyson’s ‘otherwise beautiful’ poem:

It must be manifest that if this were true, the population of the world would be at a standstill … I would suggest that in the next edition of your poem you have it read – ‘Every moment dies a man, Every moment 11/16 is born.’ … The actual figure is so long I cannot get it onto a line, but I believe the figure 11/16 will be sufficiently accurate for poetry.

I am, Sir, yours, etc.,

Charles Babbage.

Babbage’s successful cryptanalysis of the Vigenère cipher was probably achieved in 1854, soon after his spat with Thwaites, but his discovery went completely unrecognised because he never published it. The discovery came to light only in the twentieth century, when scholars examined Babbage’s extensive notes. In the meantime, his technique was independently discovered by Friedrich Wilhelm Kasiski, a retired officer in the Prussian army. Ever since 1863, when he published his cryptanalytic breakthrough in Die Geheimschriften und die Dechiffrir-k-unst (‘Secret Writing and the Art of Deciphering’), the technique has been known as the Kasiski Test, and Babbage’s contribution has been largely ignored.

And why did Babbage fail to publicise his cracking of such a vital cipher? He certainly had a habit of not finishing projects and not publishing his discoveries, which might suggest that this is just one more example of his lackadaisical attitude. However, there is an alternative explanation. His discovery occurred soon after the outbreak of the Crimean War, and one theory is that it gave the British a clear advantage over their Russian enemy. It is quite possible that British Intelligence demanded that Babbage keep his work secret, thus providing them with a nine-year head start over the rest of the world. If this was the case, then it would fit in with the long-standing tradition of hushing up codebreaking achievements in the interests of national security, a practice that has continued into the twentieth century.

From Agony Columns to Buried Treasure

Thanks to the breakthroughs by Charles Babbage and Friedrich Kasiski, the Vigenère cipher was no longer secure. Cryptographers could no longer guarantee secrecy, now that cryptanalysts had fought back to regain control in the communications war. Although cryptographers attempted to design new ciphers, nothing of great significance emerged during the latter half of the nineteenth century, and professional cryptography was in disarray. However, this same period witnessed an enormous growth of interest in ciphers among the general public.

The development of the telegraph, which had driven a commercial interest in cryptography, was also responsible for generating public interest in cryptography. The public became aware of the need to protect personal messages of a highly sensitive nature, and if necessary they would use encryption, even though this took more time to send, thus adding to the cost of the telegram. Morse operators could send plain English at speeds of up to 35 words per minute because they could memorise entire phrases and transmit them in a single burst, whereas the jumble of letters that make up a ciphertext was considerably slower to transmit, because the operator had to continually refer back to the sender’s written message to check the sequence of letters. The ciphers used by the general public would not have withstood attack by a professional cryptanalyst, but they were sufficient to guard against the casual snooper.

As people became comfortable with encipherment, they began to express their cryptographic skills in a variety of ways. For example, young lovers in Victorian England were often forbidden from publicly expressing their affection, and could not even communicate by letter in case their parents intercepted and read the contents. This resulted in lovers sending encrypted messages to each other via the personal columns of newspapers. These ‘agony columns’, as they became known, provoked the curiosity of cryptanalysts, who would scan the notes and try to decipher their titillating contents. Charles Babbage is known to have indulged in this activity, along with his friends Sir Charles Wheatstone and Baron Lyon Playfair, who together were responsible for developing the deft Playfair cipher (described in Appendix E). On one occasion, Wheatstone deciphered a note in The Times from an Oxford student, suggesting to his true love that they elope. A few days later, Wheatstone inserted his own message, encrypted in the same cipher, advising the couple against this rebellious and rash action. Shortly afterwards there appeared a third message, this time unencrypted and from the lady in question: ‘Dear Charlie, Write no more. Our cipher is discovered.’

In due course a wider variety of encrypted notes appeared in the newspapers. Cryptographers began to insert blocks of ciphertext merely to challenge their colleagues. On other occasions, encrypted notes were used to criticise public figures or organisations. The Times once unwittingly carried the following encrypted notice: ‘The Times is the Jeffreys of the press’. The newspaper was being likened to the notorious seventeenth-century Judge Jeffreys, implying that it was a ruthless, bullying publication which acted as a mouthpiece for the government.

Another example of the public’s familiarity with cryptography was the widespread use of pinprick encryption. The ancient Greek historian Aeneas the Tactician suggested conveying a secret message by pricking tiny holes under particular letters in an apparently innocuous page of text, just as there are dots under some letters in this paragraph. Those letters would spell out a secret message, easily read by the intended receiver. However, if an intermediary stared at the page, they would probably be oblivious to the barely perceptible pinpricks, and would probably be unaware of the secret message. Two thousand years later, British letter-writers used exactly the same method, not to achieve secrecy but to avoid paying excessive postage costs. Before the overhaul of the postage system in the mid-1800s, sending a letter cost about a shilling for every hundred miles, beyond the means of most people. However, newspapers could be posted free of charge, and this provided a loophole for thrifty Victorians. Instead of writing and sending letters, people began to use pinpricks to spell out a message on the front page of a newspaper. They could then send the newspaper through the post without having to pay a penny.

The public’s growing fascination with cryptographic techniques meant that codes and ciphers soon found their way into nineteenth-century literature. In Jules Verne’s Journey to the Centre of the Earth, the decipherment of a parchment filled with runic characters prompts the first step on the epic journey. The characters are part of a substitution cipher which generates a Latin script, which in turn makes sense only when the letters are reversed: ‘Descend the crater of the volcano of Sneffels when the shadow of Scartaris comes to caress it before the calends of July, audacious voyager, and you will reach the centre of the Earth.’ In 1885, Verne also used a cipher as a pivotal element in his novel Mathias Sandorff. In Britain, one of the finest writers of cryptographic fiction was Sir Arthur Conan Doyle. Not surprisingly, Sherlock Holmes was an expert in cryptography and, as he explained to Dr Watson, was ‘the author of a trifling monograph upon the subject in which I analyse one hundred and sixty separate ciphers’. The most famous of Holmes’s decipherments is told in The Adventure of the Dancing Men, which involves a cipher consisting of stickmen, each pose representing a distinct letter.

On the other side of the Atlantic, Edgar Allan Poe was also developing an interest in cryptanalysis. Writing for Philadelphia’s Alexander Weekly Messenger, he issued a challenge to readers, claiming that he could decipher any monoalphabetic substitution cipher. Hundreds of readers sent in their ciphertexts, and he successfully deciphered them all. Although this required nothing more than frequency analysis, Poe’s readers were astonished by his achievements. One adoring fan proclaimed him ‘the most profound and skilful cryptographer who ever lived’.

Figure 19 A section of the ciphertext from The Adventure of the Dancing Men, a Sherlock Holmes adventure by Sir Arthur Conan Doyle.

In 1843, keen to exploit the interest he had generated, Poe wrote a short story about ciphers, which is widely acknowledged by professional cryptographers to be the finest piece of fictional literature on the subject. The Gold Bug tells the story of William Legrand, who discovers an unusual beetle, the gold bug, and collects it using a scrap of paper lying nearby. That evening he sketches the gold bug upon the same piece of paper, and then holds his drawing up to the light of the fire to check its accuracy. However, his sketch is obliterated by an invisible ink, which has been developed by the heat of the flames. Legrand examines the characters that have emerged and becomes convinced that he has in his hands the encrypted directions for finding Captain Kidd’s treasure. The remainder of the story is a classic demonstration of frequency analysis, resulting in the decipherment of Captain Kidd’s clues and the discovery of his buried treasure.

Although The Gold Bug is pure fiction, there is a true nineteenth-century story containing many of the same elements. The case of the Beale ciphers involves Wild West escapades, a cowboy who amassed a vast fortune, a buried treasure worth $20 million and a mysterious set of encrypted papers describing its whereabouts. Much of what we know about this story, including the encrypted papers, is contained in a pamphlet published in 1885. Although only 23 pages long, the pamphlet has baffled generations of cryptanalysts and captivated hundreds of treasure hunters.

The story begins at the Washington Hotel in Lynchburg, Virginia, sixty-five years before the publication of the pamphlet. According to the pamphlet, the hotel and its owner, Robert Morriss, were held in high regard: ‘His kind disposition, strict probity, excellent management, and well ordered household, soon rendered him famous as a host, and his reputation extended even to other States. His was the house par excellence of the town, and no fashionable assemblages met at any other.’ In January 1820 a stranger by the name of Thomas J. Beale rode into Lynchburg and checked into the Washington Hotel. ‘In person, he was about six feet in height’, recalled Morriss, ‘with jet black eyes and hair of the same color, worn longer than was the style at the time. His form was symmetrical, and gave evidence of unusual strength and activity; but his distinguishing feature was a dark and swarthy complexion, as if much exposure to the sun and weather had thoroughly tanned and discolored him; this, however, did not detract from his appearance, and I thought him the handsomest man I had ever seen.’ Although Beale spent the rest of the winter with Morriss and was ‘extremely popular with every one, particularly the ladies,’ he never spoke about his background, his family or the purpose of his visit. Then, at the end of March, he left as suddenly as he had arrived.

Figure 20 The title page of The Beale Papers, the pamphlet that contains all that we know about the mystery of the Beale treasure.

The Beale Treasure – History of a Mystery by Peter Viemeister.

Two years later, in January 1822, Beale returned to the Washington Hotel, ‘darker and swarthier than ever’. Once again, he spent the rest of the winter in Lynchburg and disappeared in the spring, but not before he entrusted Morriss with a locked iron box, which he said contained ‘papers of value and importance’. Morriss placed the box in a safe, and thought nothing more about it and its contents until he received a letter from Beale, dated 9 May 1822 and sent from St Louis. After a few pleasantries and a paragraph about an intended trip to the plains ‘to hunt the buffalo and encounter the savage grizzlies’, Beale’s letter revealed the significance of the box:

It contains papers vitally affecting the fortunes of myself and many others engaged in business with me, and in the event of my death, its loss might be irreparable. You will, therefore, see the necessity of guarding it with vigilance and care to prevent so great a catastrophe. Should none of us ever return you will please preserve carefully the box for the period of ten years from the date of this letter, and if I, or no one with authority from me, during that time demands its restoration, you will open it, which can be done by removing the lock. You will find, in addition to the papers addressed to you, other papers which will be unintelligible without the aid of a key to assist you. Such a key I have left in the hand of a friend in this place, sealed and addressed to yourself, and endorsed not to be delivered until June 1832. By means of this you will understand fully all you will be required to do.

Morriss dutifully continued to guard the box, waiting for Beale to collect it, but the swarthy man of mystery never returned to Lynchburg. He disappeared without explanation, never to be seen again. Ten years later, Morriss could have followed the letter’s instructions and opened the box, but he seems to have been reluctant to break the lock. Beale’s letter had mentioned that a note would be sent to Morriss in June 1832, and this was supposed to explain how to decipher the contents of the box. However, the note never arrived, and perhaps Morriss felt that there was no point opening the box if he could not decipher what was inside it. Eventually, in 1845, Morriss’s curiosity got the better of him and he cracked open the lock. The box contained three sheets of enciphered characters, and a note written by Beale in plain English.

The intriguing note revealed the truth about Beale, the box, and the ciphers. It explained that in April 1817, almost three years before his first meeting with Morriss, Beale and 29 others had embarked on a journey across America. After travelling through the rich hunting grounds of the Western plains, they arrived in Santa Fé, and spent the winter in the ‘little Mexican town’. In March they headed north and began tracking an ‘immense herd of buffaloes’, picking off as many as possible along the way. Then, according to Beale, they struck lucky:

One day, while following them, the party encamped in a small ravine, some 250 or 300 miles north of Santa Fé, and, with their horses tethered, were preparing their evening meal, when one of the men discovered in a cleft of the rocks something that had the appearance of gold. Upon showing it to the others it was pronounced to be gold, and much excitement was the natural consequence.

The letter went on to explain that Beale and his men, with help from the local tribe, mined the site for the next eighteen months, by which time they had accumulated a large quantity of gold, as well as some silver which was found nearby. In due course they agreed that their new-found wealth should be moved to a secure place, and decided to take it back home to Virginia, where they would hide it in a secret location. In 1820, Beale travelled to Lynchburg with the gold and silver, found a suitable location, and buried it. It was on this occasion that he first lodged at the Washington Hotel and made the acquaintance of Morriss. When Beale left at the end of the winter, he rejoined his men who had continued to work the mine during his absence.

After another eighteen months Beale revisited Lynchburg with even more to add to his stash. This time there was an additional reason for his trip:

Before leaving my companions on the plains it was suggested that, in case of an accident to ourselves, the treasure so concealed would be lost to their relatives, without some provision against such a contingency. I was, therefore, instructed to select some perfectly reliable person, if such could be found, who should, in the event of this proving acceptable to the party, be confided in to carry out their wishes in regard to their respective shares.

Beale believed that Morriss was a man of integrity, which is why he trusted him with the box containing the three enciphered sheets, the so-called Beale ciphers. Each enciphered sheet contained an array of numbers (reprinted here as Figures 21, 22 and 23), and deciphering the numbers would reveal all the relevant details; the first sheet described the treasure’s location, the second outlined the contents of the treasure, and the third listed the relatives of the men who should receive a share of the treasure. When Morriss read all of this, it was some 23 years after he had last seen Thomas Beale. Working on the assumption that Beale and his men were dead, Morriss felt obliged to find the gold and share it among their relatives. However, without the promised key he was forced to decipher the ciphers from scratch, a task that troubled his mind for the next twenty years, and which ended in failure.

In 1862, at the age of eighty-four, Morriss knew that he was coming to the end of his life, and that he had to share the secret of the Beale ciphers, otherwise any hope of carrying out Beale’s wishes would die with him. Morriss confided in a friend, but unfortunately the identity of this person remains a mystery. All we know about Morriss’s friend is that it was he who wrote the pamphlet in 1885, so hereafter I will refer to him simply as the author. The author explained the reasons for his anonymity within the pamphlet:

I anticipate for these papers a large circulation, and, to avoid the multitude of letters with which I should be assailed from all sections of the Union, propounding all sorts of questions, and requiring answers which, if attended to, would absorb my entire time, and only change the character of my work, I have decided upon withdrawing my name from the publication, after assuring all interested that I have given all that I know of the matter, and that I cannot add one word to the statements herein contained.

To protect his identity, the author asked James B. Ward, a respected member of the local community and the county’s road surveyor, to act as his agent and publisher.