The surprise move was announced a day after Tom Haines was made skipper for the opening One Day Cup match away to Lancashire.
Sussex have not yet said who will take over the reins for the conclusion of the championship season - nor who will do the jobs from next season.
The county thanked Brown for his service in the role and said he would continue to be a valuable player for them.
A Sussex statement said: "Sussex Cricket’s men’s 50-over and Championship teams will be led by new captains. Tom Haines will captain Sussex Sharks in this season’s Royal London Cup. A captain for Sussex’s remaining LV= Insurance County Championship fixtures will be announced in due course.
"Permanent appointments for the roles will be made following the conclusion of the 2021 season. Luke Wright remains Sussex Sharks T20 captain.
"Given the significant changes to the makeup of the 50-over and Championship sides over the past two seasons, the club’s Performance Cricket Committee has decided that the time is right for new leadership in these formats.
"A new captain or captains will be able to develop alongside the young, homegrown players that will form the core of Sussex’s 50-over and Championship sides in the years ahead.
"As Ben Brown leaves his captaincy roles, Sussex Cricket would like to thank him wholeheartedly for his service over the past four years. Ben has never been anything but fully committed to the role and has played an important part in guiding the club in recent years, both on and off the pitch.
"His experience will be of huge value to the dressing room and we look forward to his continued contributions as a player."