Thursday, July 19, 2012

PHP scripts executed sequentially

tl;dr: session_start() is a blocking call because it locks the session file.

I had 10 images on a page that all used the same php file, but with different parameters. Getting an image took about one second. For whatever reason, the images were loaded one after the other instead of at the same time, which was problematic.

One thing that made me lose quite a few hours was that the http server wasn't Apache, it was Boost.Asio inside a daemon that I wrote. The web page was communicating with this daemon through that http server. I spent a lot of time debugging the daemon, trying to figure out whether a mutex was locked somewhere that would prevent multiple connections to be made at the same time.

I then started wondering whether there were situations when browsers cannot load images concurrently. There doesn't seem to be any, although there is a limit to the number of connections to the same server: 15 on Firefox (see network.http.max-connections-per-server in about:config), 6 on Internet Explorer and possibly 6 for Chrome (although I can't find much reliable information on that, and I don't feel like going through the code). Subdomains could be used to allow more concurrent connections to the same server, but I digress.

I then went back to the php scripts and started commenting stuff. When something happens that cannot explain, remove everything and see if it fixes it. If it does, uncomment the code bit by bit until it fails. After I commented session_start() on top of a php file, lo and behold, the images loaded concurrently.

I had never paid much attention to session_start(). I had always called it at the very top of some sort of common.php, along with database connections and defines and stuff. I knew sessions were writing information to a file on disk (in session.save_path), I had to look at them to find bugs in other scripts and, had I thought about that for a few seconds, I would have realized that this file has to be locked in some way to serialize writing.

And, unsurprisingly, it does. The session file is locked for writing when session_start() is called, which means that all the scripts that execute for the same session (which usually means in the same browser) block until the session is closed.

My solution was to treat the session as a mutex: lock it late and release it quickly.

There are two circumstances where a session needs to be opened: reading all the values in $_SESSION and writing a new value to it. To load all the values, simply opening and closing the session with session_write_close() works:
// common.php
load_session();

function load_session()
{
    session_start();
    session_write_close();
}
After load_session() is called, $_SESSION is filled and the session itself can be closed.

Instead of writing values directly to $_SESSION, going through a function allows for opening the session, writing the value, and then closing it again:
function set_session($n, $v)
{
    session_start();
    $_SESSION[$n] = $v;
    session_write_close();
}
There are synchronization issues: if two scripts execute concurrently and one writes values to the session, the second script will not get the new values because session_start() was called only once at the beginning. If synchronization is needed, the session has to be locked and scripts executed sequentially.

Note that a session cannot be started once something has been output, which means that set_session() should be treated like a header() call.

Lots of internets (if I had known what to search for):

Monday, April 23, 2012

Long delays using Visual C++ 11

I recently started using the beta version of Visual C++ 11 and had several problems with it, such as when starting the debugger.

After using it for a few hours, it started getting sluggish, and then ridiculously slow: closing a file would take between 20 and 30 seconds or regular hangs for several seconds when saving or navigating a file. I found a post about a huge suo file in the project directory for Visual C++ 2008. I took a look just for fun:

A 1.9gb suo file seen in Windows Explorer next to the sdf, sln and opensdf files

Unsurprisingly, deleting it fixed the problem.

I have no idea why this happened nor how to prevent it from happening again.

Wednesday, February 1, 2012

Timezone woes

Nothing much today, just some rambling.

We have a client in Novokuznetsk, Russia and we were having issues with displaying the right time since Russia stopped using daylight saving time. There's a WAMP server running over there with php 5.3.6 that has an old tz database. Upgrading to 5.3.9 fixed the issue, although there doesn't seem to be VC6 binaries compiled for the moment and there's a clear warning about not using VC9 version with apache.org binaries. The VC9 x86 thread safe binaries seem to work fine.

However, we were a bit uncertain about the actual time in Novokuznetsk (although we later confirmed it is +7.) Just for fun, I started looking around, taking the first few sites when searching for "timezone" on google:

Source Time Timezone
php 5.3.6 14:00 +6
php 5.3.9 15:00 +7
timeanddate.com 15:00 +7
worldtimezone.com 15:00 +7
google 16:00 +8

I was surprised to see google getting it wrong.

Wednesday, December 7, 2011

Visual C++ 2010 std::istringstream crash in destructor

tl;dr: You have a /vd2 switch in your project settings that you don't know about. Check your property sheets, gtkmm in particular has it in the additional options. [edit 11-dec-2011: but read the update at the bottom, it's not that simple]

I have a Visual C++ 2008 project that uses gtkmm, sqlite and curl that I wanted to port 2010. Most of the times, upgrading a project to a newer version of Visual C++ works quite well, but I got a crash immediately after the first build on a destructor for a std::istringstream. std::ostringstream also crashes in the destructor. The error message is
Unhandled exception at 0xwherever (ntdll.dll) in program.exe: 0xC0000005:
Access violation reading location 0x00000004.
right in
virtual ~basic_istringstream()
  {  // destroy the object
  }
 with this stack trace (edited for sanity)
ntdll.dll!_NtRaiseException@12() + 0x12 bytes
ntdll.dll!_NtRaiseException@12() + 0x12 bytes
msvcr100d.dll!_free() + 0x10 bytes
program.exe!std::istringstream::~istringstream() Line 469 + 0x20
program.exe!std::istringstream::`vbase destructor'() + 0x2e bytes
I also saw
ntdll.dll!_NtRaiseException@12() + 0x12 bytes
ntdll.dll!_NtRaiseException@12() + 0x12 bytes
msvcr100d.dll!__unlock() + 0x16 bytes
and
ntdll.dll!_NtRaiseException@12()  + 0x12 bytes
ntdll.dll!_NtRaiseException@12()  + 0x12 bytes
msvcr100d.dll!__free_dbg()  + 0x68 bytes
msvcp100d.dll!std::_DebugHeapDelete<_RTL_CRITICAL_SECTION>()  + 0x15 bytes
The first thing I found was a bug report on Microsoft Connect that was about the /vd2 switch, which I'd never heard of before and therefore do not use in my projects. I dismissed it quickly and moved on to not finding anything else.

After multiple rebuilds, checking dll dependencies and generally not having much of a clue, I started looking at the exact command line for my project in the property pages, but still could not find anything useful.

I then started to look at the individual property sheets, starting with gtkmm, where I obviously found this little /vd2 switch in the additional options of the command line. Funnily enough, it looks like these additional options are not shown on the "all options" text box of the project itself. Removing the switch fixed the crash.

Now, fixing a crash is always nice, but finding the reason for the crash is better, so it can be avoided in the future.

This gtkmm bug report dated from November 2004 states that Visual Studio 2003 had a problem with using dynamic_cast in constructors. The fix on Visual Studio 2005 was to use the /vd2 flag. Moving on to 2010, /vd2 broke for unknown reasons, but this is not fixable without breaking binary compatibility, which is not happening until the next major release. I haven't tested whether the /vd2 flag is still necessary (two bug reports seem to indicate it still is, as no resolution has been posted), but preliminary tests seem to be okay.

Therefore, it looks like the /vd flag was extended as a bugfix for 2005, but 2010 introduced a bug on the bugfix. Hopefully, the 2005 bug is gone or we'll need a bugfix for the bugfix. Who said programming wasn't fun?

There's not much internets on this, although there are two threads on gnome's mailing list and a bug report on googletest (where a guy closed it quite unceremoniously "as this is a compiler bug") that appear when searching specifically for "gtkmm" and "vd2".

[edit 11-dec-2011:

I just got linked in a stackoverflow question. I was wondering where that traffic came from.

My "preliminary tests" was successfully running a somewhat complex gtkmm application without /vd2. It was obviously not enough. This (adapted from the OP's code):
#include <cassert>
#include <sstream>

struct object
{
  virtual ~object() {}
};

struct base : virtual object
{
  base()
  {
    object* b = static_cast<object*>(this);
    base* d = dynamic_cast<base*>(b);
    
    assert(this == d);
  }
};

struct derived : base
{
  int i;
};

int main()
{
  derived d;
}

fails with Visual C++ 2010, but not with g++ 4.5.3. Adding /vd2 to the compiler settings fixes the problem, but then changing main() to

int main()
{
  derived d;
  std::istringstream iss;
}

again fails in istringstream::~istringstream().

What happens in base::base() is legal as per 12.7/5 in C++03 and 12.7/6 in C++11:
When a dynamic_cast is used in a constructor [..] if the operand of the dynamic_cast refers to the object under construction or destruction, this object is considered to be a most derived object that has the type of the constructor or destructor’s class.
Therefore, as long as T refers to a direct or indirect base type of U, U::U() can dynamic_cast this to T and back.

As far as I can tell, there is no way to fix this so that both dynamic_cast in a constructor and stringstreams can work. I don't even know the extent of the problem with /vd2 in Visual C++ 2010 (why only stringstreams?). Documentation on the /vd2 switch for 2010 says that
/vd supports incorrect behavior in an early version of Visual C++, and is no longer needed.
 but that is not what I and others see.]

[edit 28-jun-2012:

Although the dynamic_cast still fails miserably without /vd2, stringstreams now behave correctly on Visual C++ 2012 RC]

Saturday, July 9, 2011

Internet explorer 9 always in quirks mode for local files

While working on canui, I was trying to check some examples with internet explorer 9. These examples were on my local apache server. After getting some errors about the canvas object, I opened up the developer tools (in the sprocket icon) and noticed the "document mode" was in "quirks mode."

I first realized that I hadn't included a doctype declaration. The HTML5 doctype is simply

<!DOCTYPE html>

However, even with the doctype, the page was still staying in quirks mode. Pages like the fish test were working fine. When I noticed that my examples were working fine from sourceforge, my google searches changed from "always in quirks mode" to "quirks mode localhost" and I quickly found a KB article about compatibility view.

After feeling like an idiot for a few seconds about not finding the "compatibility view settings" item in the sprocket menu, I stopped looking at the images and started reading the actual text and understood that the Tools menu is not the sprocket menu. Pressing alt brings it up. I blame the lack of coffee at that time.

In the compatibility view settings, there is a checkbox "Display intranet sites in compatibility view" that is checked by default. Unchecking it fixes problem.

Therefore, make sure:
  1. the html page starts with <!DOCTYPE html> but without anything before it, including a BOM mark;
  2. the compatility view settings (in the Tools menu) for local files is unchecked;
  3. you're not forcing any specific document mode in the developers tools;
  4. you're not using any kind of X-UA-Compatible meta tags;
  5. the windows group policy isn't forcing compatibility mode.

Monday, July 4, 2011

Canvas user interface

I am currently developing a user interface library for the HTML5 canvas element. I named it canui and it is hosted on sourceforge.



Wednesday, May 25, 2011

Inheritance and virtual functions in javascript

Javascript has no built-in facilities for inheritance or virtual functions. Inheritance lets you define common functionality in a base class that can be extended or modified in a derived class. A virtual function is a function that can be redefined in a derived class and will correctly be called on an instance of that class. Although crude virtual functions can be done by assigning a different function to the object, calling the base class implementation becomes harder.

I've come up (although it might already be popular, I'm not entirely sure) with a scheme to create hierarchies with virtual function tables. Although it requires more typing and some discipline, it emulates inheritance and virtual functions correctly.

A class that is meant to be derived from is defined as a regular function, taking the derived class's this object as a parameter.

function inherit_base(self)
{
  self.member = 1;

  self.base__frob = function()
  {
    alert("frobbing base");
  }

  self.frob = self.base__frob;
}

function derived()
{
  inherit_base(this);
 
  this.frob = function()
  {
    this.base__frob();
    alert("frobbing derived");
  }
}

var d = new derived();
d.frob();

The "virtual function table" is defined at the end of inherit_base. The derived class can call the base's implementation while adding functionality.

If the base class can either be derived from or instantiated (that is, it is not what's commonly called abstract),
then a base class can be defined and derived from inherit_base:

function base()
{
  inherit_base(this);
}

This requires discipline: functions in the base class need to have a distinct name (here I use the base__ prefix) and be assigned to the "virtual" function's name afterwards. As with most languages, the base class implementation needs to be called manually from the derived class.