>>Which suggests that "try: except HTTPException:" will be specific enough >>as a catchall for this module.
>>The following, then, should catch everything you mentioned except the >>socket timeout:
> Your conclusion may be (almost) right in this case. I just don't like this > approach. Basically this is reverse engineering the interface from the > source at the time of writing the app.
This is using the source as documentation, there is no law against that. It's better than waiting for a closed source software mogul to benevolently give you the API documentation, which may not be 100% accurate itself.
> Even if you get it right, it may > fail next week when someone added an exception to a module.
Given how all of the exceptions raised by both modules are either (1) builtin or (2) defined in the module itself and given that all exceptions defined in these modules inheret from a single class, my money is on the fact that this approach should be forward compatible.
>>But it seems to me that working with the internet as you are doing is >>fraught with peril anyway.
> Why? It shouldn't be.
How things are and how things should be have always been 2 entirely different things. See early books by Robert J. Ringer for a more complete discussion.
James Stroud <jstr...@ucla.edu> writes: > > approach. Basically this is reverse engineering the interface from the > > source at the time of writing the app.
> This is using the source as documentation, there is no law against > that.
That's completely bogus. Undocumented interfaces in the library source are allowed to change between Python versions and across Python implementations. If you write your application according to the documented interfaces in the Python manual, it should not break when someone upgrades to the next Python release. You should not have to depend on undocumented aspects of the Python implementation which change out of sync with the application.
> How things are and how things should be have always been 2 entirely > different things. See early books by Robert J. Ringer for a more > complete discussion.
That's like saying programs shouldn't have bugs, but they often do anyway. It doesn't mean the bugs are a good thing or that they shouldn't be fixed.
Paul Rubin wrote: > James Stroud <jstr...@ucla.edu> writes:
>>>approach. Basically this is reverse engineering the interface from the >>>source at the time of writing the app.
>>This is using the source as documentation, there is no law against >>that.
> That's completely bogus. Undocumented interfaces in the library > source are allowed to change between Python versions and across Python > implementations. If you write your application according to the > documented interfaces in the Python manual, it should not break when > someone upgrades to the next Python release. You should not have to > depend on undocumented aspects of the Python implementation which > change out of sync with the application.
My suggestion was to use some common sense about the source code and apply it. I've wasted many hours programming to faulty documentation when a quick peek at the source code would have saved some serious time. How do you think I happened accross the technique?
I would wager that, in 10 years time, the urllib2 module will not raise any exceptions that don't inheret from urllib2.URLError or a built-in (i.e. documented) exception; unless, of course, someone made a change in a deliberate attempt to win said wager ;o)
>>How things are and how things should be have always been 2 entirely >>different things. See early books by Robert J. Ringer for a more >>complete discussion.
> That's like saying programs shouldn't have bugs, but they often do > anyway. It doesn't mean the bugs are a good thing or that they > shouldn't be fixed.
The original reference was to the internet, which seems vast in its sources of unforseen peril. If some gracious soul aspires to fix every bug in the internet, I invite him to be my guest.
James Stroud <jstr...@ucla.edu> writes: > My suggestion was to use some common sense about the source code and > apply it.
The common sense to apply is that if the source code says one thing and the documentation says another, then one of them is wrong and should be updated. Usually it's the source code that's wrong, but in either case, saying to ignore the documentation and follow the code, rather than repairing the discrepancy somehow, is poor advice.
On Sat, 04 Mar 2006 13:19:29 -0800, James Stroud wrote: > Why catch an error only to re-raise it?
As a general tactic, you may want to catch an exception, print extra debugging information, then raise the exception again, like this:
try: x = somedata() y = someotherdata() z = process(x, y) except Exception: print "Error!" print "Value of x: %s" % x print "Value of y: %s" % y raise
Although that's not the reason the Original Poster is doing this.
The OP is doing it because catching all exceptions masks bugs. There are certain exceptions which should be allowed through, as they indicate a bug in the OP's code. Normally the tactic is to catch only the exceptions you are interested in, and let everything else through, but the OP needs to catch all exceptions because there are rare exceptions which he can't predict in advance.
>The OP is doing it because catching all exceptions masks bugs. There are >certain exceptions which should be allowed through, as they indicate a bug >in the OP's code. Normally the tactic is to catch only the exceptions you >are interested in, and let everything else through, but the OP needs to >catch all exceptions because there are rare exceptions which he can't >predict in advance.
To add to that explanation: this is in a multithreaded ZODB-application. When an unexpected exception occurs and remains uncaught, a thread terminates, causing the thread pool to wait forever since some thread is not consuming its termination request from the queue, causing the app to not terminate, causing the ZODB to remain locked, causing other apps to fail, causing havoc on my server.
I don't mind this when it's caused by a bug in my code, since that creates the sense of urgency required to fix the bug. But it's annoying when it happens because of an I/O exception caused by some other guys bug on the other side of the planet :-)
Rene Pijlman wrote: > Steven D'Aprano: >> The OP is doing it because catching all exceptions masks bugs. There are >> certain exceptions which should be allowed through, as they indicate a bug >> in the OP's code. Normally the tactic is to catch only the exceptions you >> are interested in, and let everything else through, but the OP needs to >> catch all exceptions because there are rare exceptions which he can't >> predict in advance.
> ... This is in a multithreaded ZODB-application.... > When an unexpected exception occurs and remains uncaught, a thread > terminates, ....
At the base of the thread code, you could put
import sys, threading, logging
class MyThread(threading.Thread): def run(self): try: threading.Thread.run(self) # or whatever except: etype, error, traceback = sys.exc_info() logging.warning('Exception %s: %s seen at %s' % (etype.__name__, error, _someformat_(traceback))) _try_to_rescue_or remove_this_thread_
If the condition is infrequent enough. If not (if you run a real risk of multiple threads accessing the log simultaneously), have a queue of log messages that you feed to a single logging thread.
>Yes, and that's the Right Thing(tm) to do. Source code don't lie. Source > code don't get out of sync. So source code *is* the best documentation >(or at least the most accurate).
I could not disagree more strongly with this. Not just no, but hell no!
>Yes, and that's the Right Thing(tm) to do.
No, it is a horrible thing to do. But since the documentation of some modules is just plain horrible we sometimes have no choice.
> Source code don't lie. Source code don't get out of sync.
True but implementation details change from release to release.
> So source code *is* the best documentation (or at least the most accurate).
No, source code is the *worst possible* documentation because it makes no distinction between implementation detail and method contract. If the implementer considers the implementation to be the documentation then his/her refactoring options are significantly reduced. Typically implementers are not aware of this and they refactor anyway, breaking client code left and right.
The C++ FAQ has a nice discussion of this issue. Minimally acceptable documentation consists of the following (I think this is language independent):
PURPOSE: What does this method/function do REQUIRE: pre-conditions - What must have happened before calling this method (or restrictions on the domain of the inputs) PROMISE: post-conditions - What can you expect upon return or what exceptions can be thrown
I consider the above to be the minimal amount of documentation that is acceptable. If you have less than that, I consider the method to be undocumented. Needless to say, I consider lots of code that I see to be undocumented.
If you don't have the above, you get the problems that OP was hitting (or worse, see the C++ FAQ). I am not a huge fan of Java's ubiquitous use of checked exceptions or even of static typing but it does help supply some of the above documentation (although in a suboptimal way) that must be supplied by hand in Python. This is the dirty little secret of dynamically typed languages. It makes proper documentation even more important because method signatures supply less information.
|
