In most enterprise settings you will need to go through a proxy server to connect to external sites. In Windows the proxy settings in IE actually affects the apps on that machine. If you write a console app to test your code for example, it will work fine and able to connect to the external site (provided proxy settings in your IE are correct).

Copy the same code to SharePoint and it stops working. This is because by default SharePoint is not configured to use the proxy settings in IE. To get this to work there are 2 options.

Option 1: Modify SharePoint’s web.config to specify the proxy

• Edit the web.config of the SharePoint web app.
• Search for the defaultProxy element. It will look like below

<system.net>
    <defaultProxy />
</system.net>

• Change it to

<system.net>
    <defaultProxy>
      <proxy
        usesystemdefault="false"
        proxyaddress="http://123.456.78.9:1234"
        bypassonlocal="true"
      />
    </defaultProxy>
</system.net>

Option 2: Set the proxy in code for the HttpWebRequest object

WebProxy proxy = new WebProxy("123.456.78.9", 1234);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestUrl);
request.Proxy = proxy;

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

If your proxy requires credentials you will get the exception: (407) Proxy Authentication Required. To fix this add these lines:

if (request.Proxy != null)
{
	request.Proxy.Credentials = CredentialCache.DefaultNetworkCredentials;
}

Update regarding the (407) Proxy Authentication Required exception:

Setting the proxy’s credentials as above worked for me on my dev machine, which was a single-server farm. It did not work when I deployed the code to a multi-servers farm environment, and I kept getting the (407) Proxy Authentication Required exception. I found that wrapping the code that makes the request in a SPSecurity.RunWithElevatedPrivileges block solved the issue. Can’t explain what’s going on, but it works now :).