Below are some tools and thoughts that may be useful to people who are having difficulties trying to debug issues related to internet traffic.
There are a number of other tools (e.g. Firebug, Wireshark/ethereal, etc) that are not mentioned below. However I thought a quick one pager on some of the tools that I have used and found useful might be useful to someone else.

Tamper (Firefox plug-in)

https://addons.mozilla.org/en-US/firefox/addon/tamper-data/
This is a Firefox plugin which allows users to view the exact data they’re sending out through their browser. It’s useful for debugging traffic, as well as those times when your browser has remembered your password and you’ve forgotten your password (as you can see the password that is being sent). It also gives you the opportunity to intercept this data and change it which can also be useful for debugging purposes. The disadvantage with this tool though is that it will only work on data sent through Firefox. Fiddler2 (below) can be used to intecept traffic from a wider range of sources (e.g. iexplorer, firefox, wget, curl, … and other applications).

wget and curl

http://www.gnu.org/s/wget/
http://curl.haxx.se
These tools are very similar and are useful for sending web requests via command line. They both work in Linux and Windows (and other OSes). I was using this to help debug an issue we were having at a client site that was not allowing our newt traffic to be sent to our servers. It means you can easily change things like proxy settings, username/passwords, URLs, the posted data – without having to recompile an exe and copy it onto the client machine. I personally found curl worked better than wget. I couldn’t get the username/password information to be sent correctly with wget. There was plenty of help on the net, but it just didn’t seem to work for me for some reason. A typical example of the types of curl requests I was sending looked like this:
curl -L -x proxy_details:proxy_port –user username:password –output outuptfile.html http://website.com/method –data “user_info=stuffhere%3Bmorestuffhere%3Bextra_info=morestuffhere”
These command line tools can of course called via scripts as well if required.

Filddler 2

http://www.fiddler2.com/fiddler2/
I found this tool paricularly useful when diagnosing firewall/proxy issues as I could see exactly why various requests from curl/wget were, or were not, working.

I’m certainly not an expert on Fiddler2, and I’m not abou to try and tell you all the ins and outs of how to use this tool: firstly because I don’t know all of them, secondly because a quick Google search will help you there.
However, some of the useful features that I have found useful are:

• It can monitor traffic from differnt sources. The “Process” column in the left pane in the above image shows that curl and firefox traffic has been detected.
• The upper pane on the right shows the request details of the selected item in the left pane:
  • Headers
  • TextView
  • WebForms
  • HexView
  • Auth (useful if want to see how usernames and passwords are being handled)
  • Raw
  • XML
  • JSON
• The lower pane on the right gives the response details of the selected item in the left pane:
  • Auth
  • Caching
  • Privacy
  • Raw
  • XML
  • JSON
• Another one of the cool features of fiddler2 is that it can be used to debug encrypted traffic. It does this by installig a security certificate on your computer. Ideally you would only install this security certificate on a machine for debuggnig purposes and one that was not passing sensitive information over the internet as it poses a security risk. It should be removed from the computer once you have finished debugging (it’s easy enough to put another back in the future if requred).