James, 14th Earl of Morton married twice. He married Agatha Halyburton before 1731. Agatha was the daughter of James Halyburton. Agatha died in 1748 in Edinburgh. James had 5 sons by Agatha Halyburton, 4 of which predeceased him. After Agatha's death in 1748, James married Bridget Heathcote with whom he also had a son. James died in 1768 and the family possessions passed to his only surviving son by Agatha Halyburton.
He married Bridget Heathcote in St. James's Square, Westminster, 31/07/1755. Bridget was born circa 1722. Bridget was the daughter of John Heathcote 2nd Baronet and Bridget White. Bridget died 2/03/1805 in Lower Brook Street, London, at 82 years of age.
Presumably this family portrait, which hangs in the National Galleries, is of his first wife and their surviving children. It would seem likely that the boy is Sholto, who succeeded as 15th earl.
The Douglas Archives are less than complete on james and his family, so any help here would be welcome!