JC1,
The question you need to ask yourself is "What do I want to do?" In my opinion, there is a large difference between a technology degree and an engineering degree. With a technology degree, more emphasis is put on understanding technology itself (things like capabilities, limitations, manufacturing processes, etc). With an engineering degree, more emphasis is put on understanding the fundamentals of the science(s) (math, chemistry, physics, etc.) that drive technology. Engineers, however, are not scientists (chemists, physicists) per se, as they still need to have a good understanding of technology (i.e., how science is practically applied). So, in a way, engineers are a mix between scientists and technologists.
As a result of this, engineers generally come out of school with a solid background in science, but a limited understanding of its practical application. The good news is you build this understanding through experience, and with an engineering background you are well-equipped to build significantly on that understanding. That is because you not only know how to use a given technology, but you also understand how and why it works.
On the flip side of that, a technologist has a good understanding of technology right out of school. But he/she is not as equipped to move "beyond" the technology itself to the underlying science. As a result, upward mobility (and salary) with a technology degree is more limited and job duties may not be as interesting and challenging.
Now, to thoroughly understand science (chemistry, physics, and almost any branch of engineering), you HAVE to have a thorough understanding of calculus and differential equations. Don't think that you can get through Calc I and then never worry about calculus again! Calculus will come back to haunt you in almost every physics and engineering class you take. To be honest, you can probably get through engineering without understanding calculus, but at great cost. You'll just have to do a lot of memorizing. In other words, you'll be memorizing equations and solution methods for particular problems. With a solid understanding of the math (calc and diff eq), you'll be able to derive the equations themselves and you'll truly understand how to go through the solution process. A good professor will set up exams to test understanding. With a lazy professor, you'll be able to get by with memorization. BUT, if you go through college by the memorization process, it is highly likely that you'll get exposed at some point in your career (if you stay technical).
Wow, I'm really rambling now. To sum up, I would not say that your later engineering classes will be "harder" than the math you are taking now. However, those classes will test your understanding of concepts you learned in previous classes. If your understanding is good, the advanced classes will be much easier.
In other words, make sure you can handle the math. It will never go away.
Haf