Pretty simple, any state (as secular or as religious as you like) has an interest in doing things that perpetuate it, and keeping the birth rate above replacement perpetuates the state. Therefore it's generally understood by most people in most cultures that those who choose not to participate in being fruitful and multiplying have some other duty to assist with those who do participate.

Since you have put forward no means by which the unmarried person should contribute to society, you are advocating for special rights for singles that, if adopted by everyone, would lead to extinction. If you were once to say that singles should do something for society more than just be proud of hoarding wealth for themselves, that would have been notable.