Not only do we have no evidence of any sort of Ostara celebration, we have no good evidence of Ostara.

Ostara is a conjectural goddess. She was invented by Jacob Grimm in 1835 (Deutsche Mythologie pp. 180-182; English edition). His conjecture was based on three pieces of data:

1. English Eostre. Bede, De temporum ratione 15, gives an account of month-names in the older English calendar and gives Eosturmonath as the English equivalent of April. He states there that the month is named after a goddess, Eostre.

2. German Ostara. Ostara is the name of the Christian Easter festival in Old High German, and appears with that sense in the oldest extant German manuscript, the Abrogans (ca. 790 CE: St. Gallen Stiftsbibl. Cod. Sang. 911, f. 226 line 2). In addition, April was called ostarmanoth in Charlemagne’s calendar (Vita Caroli Magni 29). Forms derived from ostarmanoth also appear in Old Dutch, Old Saxon, and some mediaeval Slavonic languages; today, the name survives only in German Ostern, corresponding to English Easter.

3. Comparative evidence. Both Eostre and ostara appear to be cognate, reflexes of Proto-Indo-European *h₂eusṓs, ‘east, sunrise’. This root furnished the names of dawn goddesses in several pantheons: Roman Aurora, Greek Eos, Lithuanian Aušra, and Vedic Ushas (see further M. L. West, Indo-European poetry and myth (2007), pp. 217-227). There is nothing linking Eostre, Aurora, Eos, Aušra, and Ushas beyond the fact that their names come from the same linguistic root. We know essentially nothing about Eostre and Aurora, other than the fact that Eostre is likely to be connected to the matronae Austriahenae that appear in 2nd-3rd century votive inscriptions found near Cologne (Shaw, Pagan goddesses in the early Germanic world. Eostre, Hreda and the cult of matrons (2011) pp. 49-71). Eos, Aušra, and Ushas are entirely unlike one another. Their name comes from one linguistic root; the goddesses appear to have evolved independently.

On this (slender) basis, Grimm invented a pan-Germanic goddess. He was also responsible for the very tenuous hypothesis that she had anything to do with contemporary German Easter customs, such as the Easter Rabbit bringing eggs for children. The rabbit first appears in southern Germany in the late 1600s (Franck von Franckenau, Disputatione ordinaria disquirens de ovibus paschalibus p. 6); Eostre is in 8th century Northumbria (Bede); and ostara, remember, is the Old High German word for Easter.

The eggs themselves are a more widespread custom: the earliest appearance of them I know of is in late 14th century France (Du Cange, Glossarium mediae et infimae Latinitatis s.v. 'Ovum'). There are earlier references to egg blessings at Easter, but only in the context of blessing a wide variety of foods; the reference in Du Cange specifies a set phrase, Eufs de Pasques 'Easter eggs', dating to 1399. The practice of dyeing eggs is very widespread and may well have been borrowed from elsewhere, but we don't have the evidence to trace a line of descent.

So the bundle of ostara-rabbit-eggs comes from slapping three distinct things together and hoping they stick: (1) a name for Easter attested in 8th century Old High German, (2) Easter eggs attested in 14th century France, and (3) a critter that brings the eggs, first attested in 17th century southern Germany.

Up until the 19th century (and in many cases more recently too), it was customary to treat all shared cultural artefacts from a diffusionist perspective, meaning that they all derive from a single origin, in a single place and time. This was Grimm's perspective: that's why he bundled these things together to produce his imaginary pan-Germanic goddess Ostara. Diffusionist perspectives have gone out of fashion a bit now: not altogether, but enough that over the last fifty years, say, there's been resistence to that kind of bundling. The three elements of the bundle come from separate-ish places, and very separate time periods.

As for Easter itself, the Christian festival I mean: that can be traced back at least as far as the mid-2nd century Mediterranean -- not England or Germany. That's when we get testimony of a dispute between the Roman and Anatolian churches over the correct date to observe Jesus' death and (supposed) resurrection. The nature of the dispute -- known as the Quartodeciman dispute -- is a little alien to secular perspectives, but it's still good evidence that Easter was being celebrated at the time.

(The English name Easter, of course, came later: the Latin name was Pascha, and most modern languages other than English, German, and Slavic languages call it something related to that.)

Edit: added a bit about the matronae Austriahenae.