The Wireless Telegraph was invited in the 1890s and quickly spread to be the dominant method of peer-to-peer communication throughout the 19th century. As a method of communication, the Telegraph allowed for a secure, Intercontinental, high-throughput and extensible message transmission. Toward the end of its popularity, Telegraphs were a broadcast protocol (one sender to many recipients), and their utility skyrocketed. The design considerations and engineering of the Telegraph are seen in contemporary messaging systems, especially Apache Pulsar. This talk covers the design considerations required to enabled globally distributed, low-latency applications to run on top of Apache Pulsar. From the storage model to replication and deployment, I explore each of these design decisions and how they contrast with the design of wireless telegraphy. The talk concludes with a demo of Apache Pulsar functions and Pulsar SQL, displaying the extensibility of Pulsar. Attendees can expect to learn a brief history of wireless telegraphy and a deep understanding of the design of Apache Pulsar.
Jowanza is a software engineer located in Salt Lake City, Utah and currently employed by Pluralsight. Aside from working apache pulsar he enjoys Cycling, Model Rocketry and not Snowboarding.
In his talk Jowanza discussed the history of the telegraph and used that as a way to introduce apache pulsar.
The telegraph allowed Victorian era users to send what is equivalent to the modern day txt/sms message. Much like writing a letter, the telegraph allowed a person to send a message to a recipient. Unlike a letter, however this transmission was sent electronically using a system of wires. This allowed a user to send a message quickly from one coast of the united states to the other, or even transcendentally to countries such as Britain.
Although it provided many benefits, telegraphs suffered from a number of problems which included the following.
at-least onceguarantee, which usually involved resending the message.
Apache pulsar is a distributed pub-sub system that is part of the apache software foundation and originally developed by yahoo. It is designed to be geographically distributed between different data centers and regions.
Apache pulsar has three main components.
Pulsar addresses many of the problems that affected the original telegraph and provides solutions in most cases.
exactly-oncedelivery. Exactly once adds more latency than the former so you should examine your use cases before deciding to enabling this guarantee on a topic.
Pulsar supports a number of stream features that can be added to the cluter, like lego bricks. Pulsar-sql allows users to perform sql functions on topics. This will work even when the topic is stored across multiple systems such as s3 and glacier.
Pulsar alternatively supports Pulsar functions, which are native serverless functions. Currently functions can only have one input topic and one output topic. These functions run on the pulsar nodes, though docker support has been added recently. Currently java, python and go are the only supported languages.