Damn Newsboy Hat

I am about to admit defeat, which I don't know that I ever have done with a hat. Scarves yes, I get sick of fussy lace patterns and all the fumbling and stuff. No, I am not going to happy about it but I'm not beating my head against a wall either.
I made a rib stitch band about an inch wide, then increased every stitch, so there were probably 160 stitches or close to it. Knit about 8 inches and then decreased with a swirl pattern, and the minute I had it off the needles I hated it. unraveled it at once, back to the rib stitch. I increased on the purl stitches so there are probably 120 stitches and I'm not sure what I think yet. I'm decreasing, stitch markers every 20, and I'll probably finish it tonight. Some projects are just more of a challenge than others.
I'm not sure if I can do a brim. I don't know, it might be a big ball of yarn in the morning.