CookieContainer domain handling bug fix

CookieContainer has a bug on handling domain name here
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=478521
and Microsoft decided not to fix it in .NET 2.0/3.0/3.5.

I want to inspect the problem and want to do my own hack to solve this issue in .NET earlier than 4.0.

At first, there is three object related to cookie management in this issue, Cookie, CookieCollection and CookieContainer. Cookie is the cookie object, CookieCollection is just simply the collection of cookies and CookieContainer is a Cookie or CookieCollection container that will manage domain uri, path and expiry.

Now the issue resides in CookieContainer which not handle domain properly.

Domain handling is important because it tells client side browser or processor which cookie is visible to the client side code and can be sent back to the server. The theory is something like this:

Web from different domain can't access cookie which is belong to other domain. For example http://www.yahoo.com can't see all cookies from google.com.
Sub domain can access cookies for the sub domain and all its parent sub domain and domain itself. For example http://groups.google.com can see cookies from groups.google.com and .google.com.
Parent sub domain or domain itself can't access cookies from any of its child sub domain. So http://domain.com can't see cookies from sub.domain.com.

However I have a little confusing about the dot at the beginning as the rfc2019 standard like .domain.com and .groups.domain.com compare to non-dot domain.

Back to the issue, CookieContainer has three interested method for me to inspect. It is Add, GetCookies and SetCookies.

To properly inspect CookieContainer object, I should not trust to all these three methods instead, I want to use reflection to see what happened inside the object.
After some reflection studuying, I reveal this fields:

CookieContainer cc has m_domainTable field

Hashtable m_domainTable
Hashtable Key: string domain
Hashtable Value: PathList pathList

PathList pathList has m_list field

SortedList m_list
Key: string path
Value: CookieCollection colCookies

So from the CookieContainer object, we can get the Cookie object by this path: domain string > path string > cookies. All cookies are grouped by path and domain.
This is the code that helping me to study CookieContainer object, I get from http://channel9.msdn.com/forums/TechOff/260235-Bug-in-CookieContainer-where-do-I-report/ Thanks to JohnWinner:

public List GetAllCookies(CookieContainer cc)
{
List lstCookies = new List();

Hashtable table = (Hashtable)cc.GetType().InvokeMember("m_domainTable", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.GetField | System.Reflection.BindingFlags.Instance, null, cc, new object[] { });

foreach (object pathList in table.Values)
{
SortedList lstCookieCol = (SortedList)pathList.GetType().InvokeMember("m_list", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.GetField | System.Reflection.BindingFlags.Instance, null, pathList, new object[] { });
foreach (CookieCollection colCookies in lstCookieCol.Values)
foreach (Cookie c in colCookies) lstCookies.Add(c);
}

return lstCookies;
}

Now I want to test all Add, GetCookies and SetCookies methods if it doing right.

Testing Add method:

This are the code I use to test it:

Uri u = new Uri("http://sub.domain.com");
Cookie c1 = new Cookie("test1", HttpUtility.UrlEncode("www.domain.com"), "/", "www.domain.com");
Cookie c11 = new Cookie("test11", HttpUtility.UrlEncode(".www.domain.com"), "/", ".www.domain.com");
Cookie c2 = new Cookie("test2", HttpUtility.UrlEncode("sub.domain.com"), "/", "sub.domain.com");
Cookie c21 = new Cookie("test21", HttpUtility.UrlEncode("sub.domain.com"), "/", "sub.domain.com");
Cookie c22 = new Cookie("test22", HttpUtility.UrlEncode(".sub.domain.com"), "/", ".sub.domain.com");
Cookie c3 = new Cookie("test3", HttpUtility.UrlEncode(".domain.com"), "/", ".domain.com");
Cookie c31 = new Cookie("test31", HttpUtility.UrlEncode(".domain.com"), "/", ".domain.com");
Cookie c4 = new Cookie("test4", HttpUtility.UrlEncode("domain.com"), "/", "domain.com");
CookieContainer cc = new CookieContainer();

I test add the cookie by this,

cc.Add(c2);
cc.Add(c21);
cc.Add(c3);
cc.Add(c31);
cc.Add(c4);

and this

cc.Add(u, c2);
cc.Add(u, c21);
cc.Add(u, c3);
cc.Add(u, c31);
cc.Add(u, c4);

Note that c1 and c11 has been removed from testing Add(Uri, Cookie) and Add(Uri, CookieCollection) since using uri sub.domain.com will throw an exception The 'Domain'='www.domain.com' part of the cookie is invalid. I think the www consider a diffrent 'sub domain' which is not related to the sub domain (sub.domain.com).
Maybe the www.domain.com cookie used to let the cookie visible in the domain.com but not in its sub domain.

Checking the table, lstCookieCol, colCookies and c object in the GetAllCookies method above, I can conclude this:

Add and SetCookie method:

Add(Cookie) vs Add(Uri, Cookie) overloads:
Hashtable m_domainTable key store in different way. The key is domain name.
Add(Cookie) stored in domain key which is direct get from domain property of the cookie.
Add(Uri, Cookie) stored in domain key which are use the rfc2109 standard, all domains will start with dot. So all cookies with domain .sub.domain.com and sub.domain.com will merge into .sub.domain.com hashtable key. Note that this modification of domain name only effect to hashtable domain key and not domain property in the cookie. Domain property in cookie will remain unchange.

I get this result by checking table.Keys value. Another two Add overloads Add(CookieCollection) and Add(Uri, CookieCollection) will result the same like Add(Cookie) and Add(Uri, Cookie) respectively. Finction .SetCookie(Uri, cookieHeader) also use .Add() internally.

So now we have two different way on how the cookie stored in the CookieCollection object.
#1 The Key will be use direct from domain property of the cookie. This is related to Add(Cookie) which I think a BUG.
#2 The Key will start with dot. The dot will added in the beginning of domain property of the cookie if it doesn't have. This is related to Add(Uri, Cookie)

In my point of view #1 is a bug because it store .sub.domain.com and sub.domain.com in different group but logically we need the both cookies in the sub visible to http://sub.domain.com.

GetCookies method:

During inspecting Add method overloads, I also check the CookieCollection returned from cc.GetCookies(u).
To make easier to check the result I use this code (because List have better debugging visualization than CookieCollection):

List lstCookies = MyGetCookies(cc, new Uri("http://sub.domain.com"));

public List MyGetCookies(CookieContainer cc, Uri u)
{
List lstCookies = new List();
CookieCollection colCookies = cc.GetCookies(u);
foreach (Cookie c in colCookies)
lstCookies.Add(c);
return lstCookies;
}

Using this .GetCookies(Uri), retrieving cookies which is specific in #1 and #2 will have different result. Both result are not correct one and it resides into two different problems. Problem #1 and Problem #2 related to #1 and #2 respectively.

Problem #1: It can't retrieve cookies for current sub domain start with dot and from parent domain not start with dot. So I just get cookie c2, c21, c3 and c31.
Problem #2: It can't retrieve cookies for current sub domain since the sub domain was added a dot at the beginning. I just get cookie c3, c31 and c4.

I aspect all 6 cookies should be retrieved. After a few days, I just realize the Problem #1 and Problem #2 is the same thing on how GetCookies retrieve the cookies. It can't retrieve cookies for current sub domain start with dot and from domain not start with dot. Since Problem #2 use .Add(Uri, Cookie) and all domain key will start with dot, so it can only retrieve all parent domain.

Remember back, we have 2 domain keys .sub.domain.com and .domain.com. So when Uri is http://sub.domain.com. Only all cookies in .domain.com. can be retrieve. To make the GetCookies can retrieve the current sub domain in .sub.domain.com key, we need to make another key which is don't have dot at the beginning. It is sub.domain.com. That means we should have this 2 keys sub.domain.com and .domain.com.

Consider that the sub domain has another sub. So sub1.sub.domain.com need .sub.domain.com key in order to retrieve the cookie. So I can conclude that we need to make all domain have dot and non-dot version. Thus GetCookies can retrieve cookies from the current sub domain and all its parent.

Since using Add(Uri, Cookie) method only generate dot domain key, we need to copy it to a new non-dot domain key. Here is the function to fix it.

private void BugFix_CookieDomain(CookieContainer cookieContainer)
{
Hashtable table = (Hashtable)_ContainerType.InvokeMember("m_domainTable",
System.Reflection.BindingFlags.NonPublic |
System.Reflection.BindingFlags.GetField |
System.Reflection.BindingFlags.Instance,
null,
cookieContainer,
new object[] { });
ArrayList keys = new ArrayList(table.Keys);
foreach (string keyObj in keys)
{
string key = (keyObj as string);
if (key[0] == '.')
{
string newKey = key.Remove(0, 1);
table[newKey] = table[keyObj];
}
}
}

Thanks to SalarSoft that solve this bug fix. I get it in the source of ASProxy. This is an assumption that method Add(Uri, Cookie) is always used and dot domain key created. To be better coding you can modify this code to make sure dot domain key always mirror to non-dot domain key vice versa, then you can use Add(Cookie) method.

As a conclusion CookieContainer can be fix by using this function with this simple two conditions:
- Don't use Add(Cookie), always use Add(Uri, Cookie)
- Call BugFix_CookieDomain each time you add cookie or before you retrieve it using GetCookies or before system use the container.

CallMeLaNN

Modifying CSS in Javascript

Modify CSS class/selector value of property/key by using Javascript without changing for each element style.

In some cases, we need to change a CSS property for a CSS class to make all elements uses the class will take into effect for the property changed. If you are able to author and modify the CSS and the web page. You can use two or more classes that refer to the same styling but little different change to its property. So whenever you want to change the styling, just swap class name for any element you want.

Let say you have a table with all rows have class named .RowStyle. So the class may look like this:

.RowStyle
{
  color: #000000;
  ...
  text-align: center;
}

Now you just want to change only a color property for all rows. So you need to define new class that have nearly same properties like this:

.RowStyleRed
{
  color: #FF0000;
  ...
  text-align: center;
}

Then you need to find all rows and replace all class RowStyle with RowStyleRed because the reason you don't need to change back to the old class and still assume both class refer to the same styling but just changing a little bit.

In this case, it is good if there is an simple way to change a property is specific class so we don't need to find back all element that use the class. That the thing I want to discover.

My Situation:

In my situation, I have no power to change it. For example, like this blog, I want to change the height of Reaction features (I rename it as Feedback as shown bellow). It was defined in the ".reactions-iframe". The CSS classes is auto-generated just like the HTML and JavaScript.

This is the original properties of the class:

.reactions-iframe
{
  -moz-background-clip:border;
  -moz-background-inline-policy:continuous;
  -moz-background-origin:padding;
  background:transparent none repeat scroll 0 0;
  border:0 none;
  height:2.3em;
  width:100%;
}

I try to add the class again at the bottom of the HTML to overwrite the height like this:

.reactions-iframe
{
  height:10em;
}

However it is not overwrite the height only but the entire class. Thus, all other properties are removed. So I try to find the solution by modifying CSS class properties in JavaScript. Note that changing the value in CSS class property is not same like changing style for a DOM element which is just for that element only. Change in CSS class will take effect to all elements that using the class.

The Solution:

I found that there is no easy way to do. I don't want to use jQuery in my case since I just want to solve this CSS problem only. This post, Altering CSS Class Attributes with JavaScript give me an idea on how
possible and cross-browser compatible to modify CSS property by using JavaScript. No simple command like changing style of an element like document.getElementById('mydiv').style.height = '5.5em'. Everything is under document.styleSheets and there is a different version depend on browser. document.styleSheets[i].cssRules is used by FireFox while document.styleSheets[i].rules is used by IE where i is the style sheet document index number need to enumerate.

To make it easier we can specify document.styleSheets[i][cssRules] where cssRules is a string variable that can be "rules" or "cssRules" depend on browser. It will return an array of selector or class defined in the style sheet document. So document.styleSheets[i][cssRules][i] will return an object for single selector or class. Not so fast, a lot things to know. document.styleSheets[i][cssRules][i].selectorText is the selector or class name and document.styleSheets[i][cssRules][i].style['height'] will give you access to the height property just for example.

Here is the function:

function SetCSS(theClass, property, value) {
            var cssRules;
            var added = false;

            // Find the compatibility only one time.
            if (document.styleSheets.length > 0) {
                if (document.styleSheets[0]['rules']) {
                    cssRules = 'rules'; // IE compatible
                } else if (document.styleSheets[0]['cssRules']) {
                    cssRules = 'cssRules'; // FireFox compatible
                } else return; // unknown compatibility
            } else return; // no style sheet found

            for (var i = 0; i < document.styleSheets.length; i++) {
                for (var j = 0; j < document.styleSheets[i][cssRules].length; j++) {
                    if (document.styleSheets[i][cssRules][j].selectorText == theClass) {
                        if (document.styleSheets[i][cssRules][j].style[property]) {
                            document.styleSheets[i][cssRules][j].style[property] = value;
                            added = true;
                            break;
                        }

                    }
                }
                if (added) {
                    break; // break the outer for loop
                }
                else {
                    if (document.styleSheets[i].insertRule) {
                        document.styleSheets[i].insertRule(theClass + ' { ' + property + ': ' + value + '; }', document.styleSheets[i][cssRules].length);
                    } else if (document.styleSheets[i].addRule) {
                        document.styleSheets[i].addRule(theClass, property + ': ' + value + ';');
                    }
                }
            }
        }

This function need an enhancement since setting CSS each time will enumerate in the style sheet documents. Using like a hash table for lookup will make the enumeration faster since class name or selector is unique for a page and we can map the name to the style sheet document index. I use this function successfully in my local. However, using this function in other than localhost like in the blogger will throw an error Security error code 1000 NS_ERROR_DOM_SECURITY_ERR in FireFox. IE7 has no problem. I still inspecting why this happened. FireFox dont allow this way. I fustraited, the hole day I wasting for this endless solution. Quickly I change to the other idea.

The Real Solution:

We can't modify the defined class instead of we need to change for each object. We can reduce performance by modifying the class since it will update for all object that use the class but the allowable ways is just change the style the objects. Since I lost my day now I need the fast solution by just change it for each object.

Here is the code to change height of 'iframe' that use class 'reactions-iframe':

function SetReactionsHeight(Height) {
  var iframes = document.getElementsByTagName('iframe');
  for (var i in iframes) {
    if (iframes[i].className == 'reactions-iframe') {
      iframes[i].style.height = Height;
    }
  }
}
SetReactionsHeight('5.5em');

I dont like this solution since it will enumerate all element (if we dont know the tag name) and will cause performance hit. To enumerate for each element available in the html, use document.getElementsByTagName('*') like in this post  document.getAllElements() - or equivalent.
However it is an easy solution for simple page. If you plan using JQuery $('.reactions-iframe').css('height','5.5em') is the answer as stated How can I change the css class rules using jQuery. It just use 45 letters in single line of code to do the same like the real solution.

Javascript minify consern

Minify Javascript/CSS/HTML/XML file to make it smaller in size and fast download.

In the most recent days, I was dealing with big JavaScript code and using DotNetNuke that also use quite large JS file. I realize, it will eat my bandwidth but I just let it happened until I use jQuery. Downloading and using jquery.min.js realizing to make all my js file minified.

Before this I found JS file that minified but that time I am not realizing it will save file size about half of it. The term minify, minifying or minified is for JS with removed whitespace and comment.

I am interested with this kind of ways and here is my reading so far:

Combine, minify and compress JavaScript files to load ASP.NET pages faster
JSMin
Automatically minify and combine JavaScript in Visual Studio
Optimize WebResource.axd and ScriptResource.axd

There is so many tools out there but I want to implement it either pre-minified or automatically at runtime and combined with exception and disable when debugging in my ASP.NET application based on my options. Based on my readings, I can't find the best tool or code that implement both in one solution to make easier for developer like me.

I just have an idea to create new SourceForge project that will grab all the idea and put it into one solution.

Moreover I wanted to have this features also:
- There is a standalone GUI/command line tools to minify and/or compress included.
- The tools can be integrated to popular SDK like VisualStudio to minify and/or compress during compilation.
- Option to combine all JS files into one or not.
- Option to include or exclude one or a few JS files in the combination (Sometimes we need to seperate the JS files rather than combine it all and put in the header and make page loading later even JS script downloading become faster).
- Option to include or exclude one or a few JS files to be minify and/or compress (because maybe we want to debug one of the JS file).
- Option to disable when debugging.
- Put the minifier and compressor engine in seperate library for easier upgrade either this tool or the engine.
- * the best thing * One simple step just to add http handler or module to minify and/or compress on the fly each JS files and no need to modify anything.
- Support a few more MIME type like JS, CSS, html and xml.

Save an image in WebBrowser into a file

Save an image in WebBrowser into a file.

First of all, I am forced to share here because very hard to solve my problem and to find the solution. But the problem is simple, How to "Save an image or picture in WebBrowser into a file". It is hard for me because GDI+ is not my expertise.

About a month I searching this answer, googling around until I give up and try again, give up and try again a few time. What the most I frustrating of is after I found the solution, it is very simple. Hm...OK, at least I found it.

I am using WebBrowser to browse a site but it is not visible by user. I just want to show a picture in the page to user as a part of process like register. I don't want to use HttpWebRequest to get the image because in some cases, there is an image that may change randomly like advertisement. Also, it is out of a part of the process that need that image. In other words, I need the image from the WebBrowser not from URL itself.

To grab the image from WebBrowser IHTMLElementRender is used. It implement DrawToDC() that expose the WebBrowser to draw image to the other context. Now the image is ready to draw. It make me smile but after that I facing two big trouble.

Most of site I found, get the image from WebBrowser and show it to the PictureBox by using CreateGraphic.

Dim g As Graphics = Me.PictureBox1.CreateGraphics
Dim hdc As IntPtr = g.GetHdc
render.DrawToDC(hdc)
g.ReleaseHdc(hdc)
g.Dispose()

The first trouble is this will just a temporary image only. I can't save because DrawToDC() is just draw the image to the graphic that get from PictureBox.CreateGraphic. The image if on top of the PictureBox, not stored in the PictureBox.Image. The image can be missing if PictureBox refresh itself.

The second trouble, I wan to make my PictureBox and/or y Form hidden. After a few sample I found, I try to use New Bitmap to store the image. Then use BitBlt function to copy Graphic from PictureBox to the new bitmap. Now it is ok. I can save the image by using Bitmap.Save method. However when I try to hide or invisible the PictureBox, the saved image is an empty image. I take so many time here to trying the thing that I don't like to. The GDI+ and all its messy functions.

After a month I found this site. It tell how to capture overall web page image from IE to a file. I try to follow the algorithm on how to store the image without using CreateGraphic, BitBlt and so forth. Now I did it =)

Here is the final code:
Reference to Microsoft.mshtml needed. elem is the image element (img tag).

Private Sub StoreImage(ByVal elem As HtmlElement, ByVal FilePath As String)

If elem IsNot Nothing Then
Dim oelem As IHTMLElement = CType(elem.DomElement, IHTMLElement)
If oelem IsNot Nothing Then
Dim render As IHTMLElementRender = CType(oelem, IHTMLElementRender)
If render IsNot Nothing Then

Dim Temp As Bitmap = Nothing
Dim gTemp As Graphics = Nothing
Dim hdcTemp As IntPtr

Try
Temp = New Bitmap(elem.OffsetRectangle.Width, elem.OffsetRectangle.Height)
gTemp = Graphics.FromImage(Temp)
hdcTemp = gTemp.GetHdc()

render.DrawToDC(hdcTemp)

gTemp.ReleaseHdc(hdcTemp)
gTemp.Flush() ' please flush before save it
gTemp.Dispose()
gTemp = Nothing

Temp.Save(FilePath)

Catch ex As Exception
Dim a = ex.ToString()
Finally
If gTemp IsNot Nothing Then
gTemp.ReleaseHdc(hdcTemp)
gTemp.Dispose()
gTemp = Nothing
End If
If Temp IsNot Nothing Then
Temp.Dispose()
Temp = Nothing
End If
End Try

End If
End If
End If

End Sub

See the code inside the Try. Its very simple.

Please note that I have found this code before, create new image, create graphic from it, create the hdc from the graphic and call DrawToDC. However, because I want to proper dispose it, I use Try Finally block that handle it properly. This will make the image saved as a blank image. That is because I save it immediately after DrawToDC. I can't do that since the graphic does not update the bitmap. Then don't forget to call Flush().

Yeah, Finally Flush() is what I missing. Now I should take a rest.