Just outside the Paris city limits there is the Chateau de Vinncennes which is conveniently on the both the Metro 1 and the RER A lines. I'm not sure how well-known it is- it seems to be off the tourist path since the only reason why I knew about it was because it's not that far from where we live. Also: not crowded at all despite the fact that it's really easy to get to.
This one was originally built in 1150 as a royal hunting lodge for the nearby Bois de Vinncennes, which still sprawls out as an enormous verdant parkland around it. In the 14th and 17th centuries, it was fortified, moatified and expanded upon, became a prison for a bit and now it serves as a military HQ of sorts.
You can wander around the grounds for free, or pay a few euros to get into the medieval tower and the gothic church. Nothing too interesting inside, as any furnishings or personal touches are long gone.
What makes it worth the visit is that it was eventually turned into a prison. The internal walls are covered with graffiti left hundreds of years ago by prisoners.
Notably, the Marque de Sade was imprisoned here for a spell...for "Lewd behavior at a House of Ill Repute". He wrote the heart-warming family classic "Justine" while locked up.
Aside from that literary gem, it was a little underwhelming. They seem to be doing some heavy restoration on some of the structures, so maybe it will get better. Afterwards, there are cafes across the street and a Botanic Garden at the edge of the park that made for quiet walk.