Ben Franklin, that noted Founding Father, started it all.
In 1775, the U.S. Postal Service was born. Ben Franklin, Founding Father, inventor of spectacles and deliverer of folksy homilies, also invented the library and yes, the post office. It's explicitly authorized by the U.S. Constitution, and the Postal Department, created in 1792, now employs about 700,000 workers.
Competition arose soon enough
Private parcel services began to deliver packages when complaints began that the U.S. Postal Service inflated rates. It held a monopoly over regular mail and postmasters were routinely appointed to their jobs as political favors.
In 1852, a mere 78 years or so after the invention of the post office, Wells Fargo began to provide both banking and express package delivery services.
One of the reasons parcel delivery was so important was because California gold was a hot commodity then, and other financial dealings also required a secure method of currency delivery across the country. Wells Fargo became a stagecoach business, and participated in the development of the Pony Express. Until the transcontinental rail lines were completed in 1869, Wells Fargo remained the primary player in overland traffic for both regular mail delivery and package delivery.
Going to rail
Beginning in 1869, overland service moved to rail service. This was faster and cheaper, with the "express office" part of the railroad station staffing. Packages took their place in passenger trains, and the United States Railroad Administration was formed in 1918; these services were then consolidated into a single agency, continuing after World War I as the Railway Express Agency.
Say Hello to Parcel Post
Rural customers got their packages delivered to them beginning in January of 1913 along with the regular mail. No longer did they have to make a trip to town, and thus the express office saw its demise. Rural free delivery further fueled catalog sales and as a result, package deliveries. The post office once again had a monopoly on mail, such that Wells Fargo ultimately dropped out of package delivery to focus on banking.
UPS makes its entrance
Gas and diesel powered trucks fueled the rise of UPS, first as a private courier service and then as a nationwide service after highway expansion was successful following World War II. UPS saw competition very soon, while rail-based services diminished such that the Railway Express Agency was dissolved in 1975.
Federal Express takes advantage of airmail
Although airmail first saw development in 1918, the "hub and spoke" system that was developed by Federal Express in 1973 and the deregulation that occurred in 1977 allowed FedEx to deliver small packages overnight throughout the US. The U.S. Postal Service countered with Express Mail and began to contract with Amtrak to carry – ironically – rail mail again.
In the 21st century: Shipping consolidators
Today, e-commerce demands low-cost package shipping services. Merchants now face customers who don't want to pay high shipping costs in exchange for having packages delivered to their doors. As a result, shipping consolidators step in to harness the strength of the U.S. Postal Service with its "last mile delivery" option, while they also take advantage of the advanced technologies afforded by the private carriers to lower shipping costs in total.