Brown signed a four-year contract extension with the Celtics in October of 2019, then went on to appear in (and start) 57 of the team's 72 games during the pandemic-shortened season. The athletic wing posted career-high averages in points (20.3), rebounds (6.4), assists (2.1), steals (1.1) and made three-pointers (2.3). On Dec. 11 against the Pacers, Brown dished a career-best eight assists to accompany 18 points and eight rebounds. On Christmas Day, Brown piled up 30 points, six rebounds, four assists and five three-pointers in a road win over Toronto. Two days later, he scored a career-high 34 points with nine rebounds in a home win over Cleveland. Brown twice won Player of the Week honors, first on Dec. 30 and then on Feb. 3. The Celtics were one of 22 teams to resume play in late summer at the Orlando bubble. On Aug. 2, Brown posted 30 points, six boards, a season-high six three-pointers and two steals in a win over the Trail Blazers. The Cal product started all 17 of Boston's postseason games, producing a number of highlights along the way. In Game 6 of the Eastern Conference semifinals against Toronto, Brown delivered 31 points, a postseason-career-high 16 rebounds, two dimes and two steals. All in all, he averaged 21.8 points, 7.5 boards, 2.3 assists, 2.5 three-pointers and 1.5 steals in the playoffs.

Brown entered his third NBA season with great expectations after his outstanding performances in the playoffs the year before. However, with Kyrie Irving and Gordon Hayward both healthy, Brown ended up starting just 25 times in 74 games. Nonetheless, he remained an efficient scorer for the Celtics, averaging 13.0 points along with 4.2 rebounds per game. Brown duplicated his 46.5 field-goal percentage from the year before and made 1.3 three-pointers per game, hitting them at a 34.4 percent clip. Despite the limited opportunities available to him, the Cal product still scored 20-plus points on 14 separate occasions, with 11 of those performances coming off the bench. That included his biggest game of the year, a 30-point outburst on 10-of-15 shooting against the Spurs on New Year's Eve. Brown also collected a pair of double-doubles off the bench, most impressively picking up 24 points and 10 rebounds in a Jan. 30 win over the Hornets. He went on to start all nine of the Celtics' playoff games, averaging 13.9 points on an impressive 50.6 percent shooting along with 5.8 rebounds per game. Brown's top playoff performance came in Game 3 of the first round against the Pacers, when he scored 23 points with seven boards.

Brown grew into a far larger role in his sophomore season, blossoming into one of the league's most promising young swingmen. The former No. 3 overall pick started all 70 games in which he appeared, delivering averages of 14.5 points, 4.9 rebounds and 1.0 steals per game -- blowing his rookie numbers out of the water on all counts. He upped his field-goal percentage to 46.5, thanks in large part to much-improved accuracy from three-point range -- 39.5 percent. An excellent defender, he was usually matched up against the opponent's top scoring wing, but he did plenty of scoring of his own. Brown opened the season with a 25-point game against Cleveland on Oct. 17, exceeding his personal best as a rookie right away. He went on to score 20 or more points 16 times, topping out with a 32-point game (including a career-high seven three-pointers) against the Bulls on Apr. 6. He also double-doubled thrice, including collecting a career-high 13 rebounds against the Hornets on Nov. 10. Come playoff time, an injury to Kyrie Iriving forced Brown and other young Celtics to step up. Brown became one of the team's primary scoring options, posting 18.0 points and 4.8 rebounds per game across 18 contests in the Celtics' run to the Eastern Conference Finals. He cleared 30 points twice in the first round against Milwaukee, including exceeding his regular-season best with 34 points in Game 4 of that series.

The Celtics took Brown third overall out of Cal Berkeley in the 2016 NBA Draft. He got his first start in the fifth game of the season, posting 19 points, five rebounds, two assists, three steals and a block in a loss at Cleveland. After his third start, Brown returned to a bench role. However, he again got the opportunity to start when a late-January injury to shooting guard Avery Bradley opened up space in the lineup. Ultimately, Brown started 20 times among his 78 games, averaging 6.6 points and 2.8 rebounds while making 45.4 percent of his field goals and 34.1 percent of his shots from three-point range. He set his season high in scoring in a start Jan. 27 against Orlando, racking up 20 points along with eight rebounds. Defensively, he proved to be a good fit for coach Brad Stevens' switching schemes. Brown's defense off the bench contributed to the Celtics finishing with the best regular-season record in the Eastern Conference. The Celtics made it all the way to the Eastern Conference Finals before falling to the Cavaliers in five games. Brown came off the bench for all but one of Boston's 17 playoff games, averaging 5.0 points and 2.1 rebounds per contest. He exploded for 19 points off the bench in Game 2 against Cleveland. Following the season, Brown was named to the All-Rookie Second Team.