The area highlighted on your attached map is called the Northwest Angle. The 1783 Treaty of Paris included many details about the future boundaries between the territories of Britain and the new United States that turned out not to match very well with the actual physical geography.

Among these, the treaty-makers seem to have thought that the source of the Mississippi was further north and east of it's real position. The treaty specified the boundary between the US and what would later become Canada as "north to the lake of the woods and hence west to the Mississippi".

Problem is the entire Mississippi River is too far south for a single E-W line to intercept both it and the Lake of the Woods.

You're question on the history has been answered but I'll elaborate on the modern day aspects a bit. To actually get to the angle inlet from MN you have to go through customs in Canada and then drive back into the US, there's no checkpoint going to the inlet but there is a booth at the border where you have to phone customs. You can also get there by boat, or by car over the lake in winter. There's maybe 100 or so residents and most of their income comes from walleye fishing if if I remember correctly. They actually tried to secede to Canada in the late 1990s but the local Ojibwe tribe was against it.

It's a really beautiful area and some of the best walleye fishing you'll find anywhere.