If some task comes easily to me, I usually feel like I am not trying hard enough and taking the easy road.

There are two questions here.

Is it true?
Not necessarily because it is subject to a multitude of factors that are specific to the circumstance and person.

And why does it matter?
The answer to this question is more important. The level of ease in performing a task should not determine its importance. What should influence it is the value it holds for me, measured in terms like happiness and fulfillment.

We don’t always have to struggle. Sometimes the easier path can be the right one.