Steam was an early candidate, but by the turn of the century, it had become clear to most that the gas-powered internal-combustion design was the best route to get to higher-powered, more reliable engines. The shape of those engines, however, was hardly a given. There were simply numerous paths to that end goal, and the shape of engines in the early days bore evidence of a spirit of creative experimentation that defined aviation and helped it drive down the most efficient path.
There are remarkably few dead ends, at least for engine technologies that achieved any kind of foothold in the burgeoning industry. But one such wandering attempt was the rotary engine, not to be confused with the radial engine, a design that lived a longer life than the rotary.
A rotary engine is one in which the cylinders, like on a radial engine, are arrayed about a center prop shaft. The radial layout of both designs is great for air cooling, but the basic concept behind the two types is fundamentally different. In a radial engine, the cylinders are fixed and drive a spinning crank/prop through a geared cam system. On a rotary engine, which predominated during the World War I era, the cylinders rotate along with the crank/prop shaft. To the modern ear, it sounds absolutely insane, but it worked better than any other design at the time, producing more power at lower weights thanks to weight savings in cylinder and case designs not available to fixed cylinder designs of the day.
The Gnome and Le Rhône rotary designs were developed separately, and many were built under license. The companies merged just before the start of World War I, and the engines were manufactured under license around the world. During the war, these rotary engines, which turned out 110 hp in their most common variant, were the best aircraft engines available for a number of years. Because they were produced by licensees in both Allied nations, including the United States, and Central Power countries, including Germany, the engines became standard equipment for aircraft fielded by both factions. More than 100,000 of these engines were produced by licensees in several different countries, and the majority of WWI aircraft were powered by one of these engines.
But there are a few glaring disadvantages to the type, including the need to mix lubricating oil (castor oil) with the fuel, which was a zero-retention system that was also messy and time-consuming.
One of the misconceptions about rotaries is that they are an all-or-nothing affair, and there's good reason for that belief. But in normal operations, the power output could be controlled by adjusting the fuel supply, which required sensitive adjustment, making it impractical when constant power changes were required. So, pilots learned to use the ignition cutoff (aka, blip) switch to shut off the engine's ignition, restarting it when they needed a dose of power. That is the reason for the odd "vroom, vroom" sound of a rotary engine when these planes land. There's also the issue of gyroscopic precession associated with having a heavy spinning mass on the nose of the airplane, creating handling problems, especially in heavier rotaries operating at higher power levels.
Rotaries were also limited in growth potential, as the number of cylinders was limited to the space available at the hub of the star shape, and, unlike radial engines, multi-row rotaries, while attempted, were difficult to pull off and limited in scale.
By the end of WWI, rotaries were already a dying design.