Naturally I'm going to say that Kanji is worthwhile to learn, since I took the time to do it
However, I can see the possibility of confusion in Japanese if you already learned 3,000 common characters for Chinese. Although onyomi are supposed to be the "Chinese pronunciation," my brief glimpses at Chinese language suggest that the sounds are not very similar these days. As such, this may present an extra challenge in your case.
Perhaps you might take a layered approach to Kanji. Start by focusing on the onyomi of each character. As with Chinese, there are rarely more than two pronunciations. This will allow you to pronounce perhaps 90 of all complex Japanese words, which are usually represented by two Kanji characters.
As you go along, then, you might next memorize just the most common kunyomi form of each character. I apologize for the vagueness of this suggestion; it is not always easy to know what the most common kunyomi is. Knowledge of spoken Japanese might help to guide you in this.
In time, you could memorize the less common kunyomi pronunciations, if you wish.
One way or another, if you try to learn Kanji, you will also have to memorize the pronunciation of each complex word in context. Although most of these words use two onyomi, it is not always clear which onyomi (ex: 不正 is pronounced "fusei," but 正 could be either "sei" or "shou"). Further, onyomi pronunciation can change slightly in combination (ex: 一本 is "ippon" rather than "ichihon"). Of course there are also plenty of situations where kunyomi pronunciations are used instead (ex: 取り引き is "torihiki"); although this can be obvious when words are combined with hiragana, sometimes kunyomi pronunciation does not require this (ex: 取引 may be a more common spelling of "torihiki"). The bottom line is that sometimes you can't figure out the correct pronunciation -- you just have to know it.