I'd argue that as long as he still receives his share of the profits he still has ownership of the team.
I can't see any judge buying that, ownership implies control, unless you are a minority share owner (meaning percentage, not race).
I'd also argue that the league needed to remove him from the day to day operations because his racist behavior brings any actions he takes regarding most of his employees under threat of litigation for racial discrimination and that liability would extend to the NBA as well.
The league (and when I say "the league" I mean the other owners he signed the contracts with) is not his employer. Like the owner of a small business, that owner is free to make racist decisions that carry liability that bring down his business. A more accurate analogy is that Adam Silver is actually Donald Sterling's employee, not vice versa.
The only power the league has over him is pursuant to what amounts to a series of contracts that he signed with the rest of the owners.
The league wrote up those contracts and had him sign them. However, there are many reasons why any two entities could write up a "contract" and then have it fail to be enforceable.
Yes, the NBA has very smart lawyers who undoubtedly wrote those contracts so as to make them as enforceable as they could, but at the end of the day they are writing them to favor the NBA and they are just guessing based on the closest precedents they have in court decisions. Since there has never been a case exactly like this before, it's new ground.
I see the argument the league can make. I'm just not sure if it can overcome stripping him of ownership.