Depression can happen, and when it does, it is real. But you, as the patient, are at least aware of what to look for and can seek out tools to help you recognize the symptoms and control the situation. I guess being fore-warned is also being fore-armed.
The one gating factor to your getting back to work at 3 weeks (aside from the question of whether you actually feel up to it) may be how to get to work. If you take public transportation - are you up to that mayhem? Long waits for trains/buses? If you drive, will you be released to drive yet? (Many of us weren't released to drive until 4 or 5 weeks, and might not have been up to the torso movement required to drive even at that time.) If you have a way to get a ride to and from work, of course, then these cautions don't apply.
Another thing to keep in mind is that the meds used in and after surgery may have effects on you even out several weeks. You won't know until then what, if any, effects there will be. I know that at 3 weeks I was just stopping the pain meds, and was still not up to my normal mental acuity levels. By 4 or 5 weeks, I felt sharp enough to get back to my normal range of work responsibilities, with shorter hours. (To give you an idea of my stress levels, I am Chief Financial Officer of a manufacturing company that supplies the auto industries in about 30 countries. No stress here. . . )
At this point I would keep all of the options open. Plan for the best but be prepared for less. Tell yourself you will go back at 3 weeks, but give yourself permission to take more time if needed. I told my office that I would be out about 6 weeks, but that it could be up to 8. Then when I got back at 5 weeks, we were all happy.