Someone who steals from private hands is as much of an aggressor/innocent as someone who steals from social hands.
Note that socialism is not about public (government-controlled) goods but socially owned (community/cooperative/group-controlled) goods. ie. in this case you should compare it to someone who steals from your local community (let's say.. a building where the ownership is shared) something that belongs to a group or a collective. Not necessarily to the government.
You are probably gonna argue that it's not the same to take money from someone than to steal a good. But as a matter of fact, taking money from someone is what is done in every purchase. When someone takes a good from your shop without paying for it, you are actually forcing him to give you money in exchange, in the same way as you pay taxes in exchange from being able to be part of a community with its benefits and obligations. You can always emigrate to the United Arab Emirates or any other country without taxes if you don't want to be in a community that has taxes as part of its social contract established by "we, the people".
You can't expect to have the police defend you from stealing and at the same time refuse to pay the salary of the police (which in the USA it's publicly-funded with taxes, same as the army, the sewage, the garbage collection/treatment, and other "evil" social services).