No, all languages are equally difficult to learn for a native speaker growing up immersed in one. All natural languages are equally complex, but in different ways. English is most complex phonologically, meaning it has a large group out sounds and many ways they can fit together. It's simple morphologically, meaning words don't change very much in different situations. Think about the way Spanish verbs are different depending on the person/group/thing doing the action; English verbs barely change. Similar situation for nouns; Spanish nouns follow a gender system that affects other words near them, which English does not. Comparatively, Chinese has extremely simple grammar - there's no gender, number, tense, or conjugation. But it has a complex tone system where the exact pronunciation of a word can completely change its meaning. You can look at any language and see what's simple and what's complex.

The difficulty of learning a language later in life is determined entirely by how similar it is to one's own language. So European languages are relatively easy to learn for speakers of other European languages (generally) and Chinese languages are relatively easy to learn for speakers of other Chinese languages (generally). For an English speaker (for example) the easiest languages to learn are Spanish and Norwegian/Danish/Swedish; they have simpler phonology than English and similar vocabulary and grammar. The most difficult are Japanese and some Native American languages, which are extremely grammatically different from English.