How smart are birds?

I feel awful! We've had a wet summer and I was clearing out some vines that were overtaking the side of the house. Without realizing it I pulled down a cardinal's nest. I should have known something was up because Mom and Dad cardinal were watching me the whole time. I accidentally stepped on one fledgling, then found another who was chirping. I put the baby bird back in the nest, put the nest back in the lilac tree. But - will Mom and Dad find it? I feel like crap. I didn't know birds made nests and raised babies this late in the summer.