There at least three peculiar accounts from Scotland that do appear describe Inuit who appeared well south of their usual fishing grounds.

From James Wallace, minister of Kirkwall in Orkney, written in 1688:

Sometimes about this country are seen these men they call Finnmen. In the year 1682, one was seen in his little Boat, at the south end of the Isle of Eda, most of the people of the Isle flocked to see him, and when they adventur’d to put out a Boat with men to see if they could apprehend him, he presently fled away most swiftly. And in the year 1684, another was seen from Westra, and for a while after they got few or no Fishes for they have this Remark here, that these Finnmen drive away the fishes from the place to which they come. These Finnmen seem to be some of those people that dwell about the Fretum Davis [Davis Strait] a full account of whom may be seen in the natural and moral History of the Antilles chap. 18. One of their boats sent from Orkney to Edinburgh is to be seen in the Physitians hall, with the Oar and Art he makes use of for killing Fish.

Wallace's note was published by his son, Dr James Wallace, who added a further gloss:

I must acknowledge it seems a little unaccountable how these Finn-men should come on this coast, but they most probably be driven by Storms from home, and cannot tell when they are any way at Sea, how to make their way home again; they have this advantage, that be the Seas never so boisterous their Boat being made of Fish Skins are so contrived that it can never sink, but is like a Sea-gull, swimmnig on top of the Water. His shirt he has so fastened to the Boat, that no Water can come into his Boat to do him damage, except when he pleases it, which he never does but to ease nature or when he comes ashore.

Their account gets some backing from John Brand (1701):

There are frequently Finnmen seen here upon the Coasts, as one about a year ago on Stronsa and another within these few months on Westra, a Gentleman with many others in the Isle looking on him nigh to the shore, but when any endeavour to apprehend them they flee away most swiftly; which is very strange, that one Man sitting in his little Boat, should come some hundred of leagues, from their own Coasts, as they reckon Finland to be from Orkney... The Fishers here observe that these Finalnd or Finland-men, by their coming drive away the Fishes from the Coasts. One of their boats is kept as a Rarity in the Physians Hall at Edinburgh.

Meanwhile, the Revd Francis Cantrell, writing from Aberdeen (on the east coast of Scotland, a further 200 miles to the south), had this to say about an incident that occurred there in 1728:

A canoe about seven yards long by two feet wide, which about thirty-two years since was driven into the Don [the river that flows through the city] with a man in it who was all over hairy, and spoke a language which no person there could interpret. He lived but three days, although all possible care was taken to recover him.

A second version of what is probably the same incident was written up in 1782:

A canoe, taken at sea with an Indian man in it, about the beginning of this century. He was brought alive to Aberdeen, but died soon after his arrival, and could give no account of himself. He is supposed to have come from the Labrador Coast, and to have lost his way at sea. The canoe is covered with fish skins, curiously stretched upon slight timbers, very securely joined together. The upper part of it is about twenty inches broad at the centre, and runs off gradually to a point at both ends. Where broadest, there is a circular hole, just large enough for the man to fit in, round which there is a kind of girth, about a foot high, to which he fixed himself, probably, when he did not use his oar, or paddle; which when he chose it, he stuck into some lists of skin, tied around the canoe, but slack enough, to let in the padle, and some other awkward utensils which were found stuck there. The canoe is about eighteen feet long, and slopes on both sides, but the bottom is flat for three or four inches in the middle and gradually sharpens as it approaches the extremities till it ends in a point.

The canoe this man apparently arrived in still exists and is listed in the Aberdeen Museum catalogue compiled in the 19th century as "Esquimaux Canoe, in which a native of that country was driven ashore near Belhelvie about the beginning of the eighteenth century and died soon after landing." It is clearly of Inuit design.

So, to summarise: the descriptions we have are consistent with Inuit in kayaks, and at least one kayak which appears to be clearly of Inuit design and construction still exists. The remaining puzzle is, assuming the reports do describe Inuit, how they got to a point so far south of their usual hunting grounds - by themselves, or as passengers (willing or unwilling) on some larger ship?

We know the latter type of event happened on at least one occasion involving a Danish ship, dating to 1654 and reported by Adam Olearius in his Travels of the Ambassadors sent by Frederic, Duke of Holstein, to the Great Duke of Muscovy and the King of Persia (London, 1662 & 1669):

As soon as this ship appear’d upon the coasts of Greenland, the Inhabitants set out above a hundred boats, and came to view that strange structure, which was much different from what they ordinarily saw. At first they would by no means come near it, but seeing they were entreated to come into the ship, they at last came, and in a few days were so familiar, that with their commodities, which they truckt for such toies as we had, they brought also their Wives, out of an intention to make advantage of them by another kind of Commerce, which though it be not known less elsewhere, yet is not so publickly practis’d Among them, where fornication is neither crime nor sin. The Danes thought this freedom of the Groenlanders a good opportunity to carry away some of them. The Ship being ready to set sail for its return, and the Savages coming still aboard with their Commodities, a Woman that had a great mind to a pair of knives which one of the Sea-men wore at his Girdle, offer’d him for it a Sea-Dogs skin, which the Sea-man refusing as too little, she proffer’d him a kindness into the bargain. The Sea-man had no sooner express’d his being well satisfy’d with the proffer, but she begins to untie the point (for they as well as the men wear Drawers), and would have laid herself down upon the Deck. But the Sea-man made her apprehend that he would not have all see what they did, and that she must go under Deck. The Woman, having got her Father’s leave, followed the Sea-man with two other aged women, a young Boy, and a Girl about 12 years of age, who were to be present at the consummation of the bargain. But as soon as they were down, the hatch was shut, they laid hold also of another Man, and set sail.

That said, two modern kayakers, Patrick Winterton and Mick Berwick, completed a 250-mile voyage from Stornoway to the Faroes in 2009, so perhaps the concept of independent visits made by Inuits to Orkney and the coast of Scotland is not utterly far-fetched.

For further discussion, see

Dale Idiens, 'Eskimos in Scotland c.1682-1924' in Christian Feest (ed), Indians and Europe: An Interdisciplinary Collection of Essays.

David Macritchie, 'The Aberdeen kayak and its congeners'. Proceedings of the Society of Antiquaries of Scotland, 1912.

Andrew Jennings, 'The Finnfolk' (Text of a public talk delivered at the Shetland Museum)

The Fortean Times did an article some years back where they suggest that some of the "Mermaid" sightings in Northern European waters could actually be Coastal Sami in their kayaks. (A maritime subgroup of the Sami/Lapps)

The Coastal Sami were known to range extensively across the Arctic and North Sea regions so it's not too unreasonable to suggest that some visited the British Isles.

More on the Coastal Sami here: https://www.laits.utexas.edu/sami/dieda/hist/nor-sami.htm