IoT application processor chips and camera chips

　　In recent years, artificial intelligence technology and Internet of Things technology have gradually integrated and developed. Massive data is generated and collected through the Internet of Things and stored in device terminals, edges, and clouds. The data is then intelligently analyzed through machine learning, ultimately realizing the intelligent connection of all things data action, AIoT emerged as the times required. As the core component of AIoT equipment, the importance of chips has become increasingly prominent.

　　With the popularity of the Internet of Things, artificial intelligence, and electronic terminals, SoC chips have become the mainstream direction of current integrated circuit design and development. Among smartphone application processor chips, Apple's A-series chips, Qualcomm's "Snapdragon series" chips and MediaTek's "Dimensity series" chips are all SoC chips. SoC chips mainly optimize the three core indicators of the chip "PPA" by using more advanced processes.

　　However, as Moore's Law gradually approaches its limit, the process evolution of wafer manufacturing slows down, and the design of SoC chips begins to shift to innovation in the internal architecture of the chip and innovation in packaging. In addition, according to the diverse downstream application markets, not all SoC chips need to use the most advanced processes.

　　Through the innovation of chip system architecture, SoC chips manufactured using relatively mature processes may also achieve the "PPA" that can only be obtained by advanced generation processes.

　　The main control chip of the Internet of Things intelligent terminal equipment is an SoC chip. Video and audio are the two major application directions of IoT intelligent terminal products. The main products related to video or audio-combined applications are IoT cameras. Its main forms include home cameras in smart homes, video doorbells, baby monitors, visual collection equipment in smart retail, and security cameras in smart security. , store monitors, video conferencing systems in smart offices, panoramic cameras, reversing rearview mirrors, driving recorders, visual sensors in smart cars, industrial vision systems in industrial applications, etc. Applications related only to audio include TWS headsets, Bluetooth speakers, etc. Including ISP, NPU, image signal preprocessor, system control, audio codec, analog-to-digital converter, encryption module, and numerous peripheral interfaces and human-machine interfaces.

　　IoT smart terminals also include other product forms. For example, access control and attendance in smart offices, building video intercoms, smart door locks, and control panels in smart security, code scanners in smart retail, display controllers in the industrial Internet of Things, etc. In addition, with the popularization of IoT technology, various forms of IoT products will emerge in endlessly.

　　As the main control chip of various types of IoT smart hardware, SoC chips determine the performance, complex and simple functions and price of core components of downstream application products. Its technology development trend depends on the demand for downstream application products.

　　In recent years, with the maturity and popularization of 5G, IoT, artificial intelligence, big data and other technologies, IoT intelligent hardware has been greatly improved in form, function, performance, and other aspects. Traditional multimedia processing algorithms for video and audio need to be compared with The integration of deep learning algorithms and innovation in SoC chip architecture have brought a large amount of market demand and unprecedented development opportunities to SoC chips.

　　(1) IoT camera chip

　　IoT camera products

　　IoT cameras are a new generation of cameras that combine video coding technology, wireless network transmission technology, intelligent tracking and identification technology, and cloud storage technology. After the IoT camera collects and encodes video information, it can be transmitted and stored online through wireless communication technologies such as cellular networks or Wi-Fi and is embedded with intelligent technologies such as face recognition, motion detection, night vision switching, and voice recognition interaction.

　　Compared with traditional analog cameras, a major feature of IoT cameras is the added wireless network access function, which converts digital video signals into data streams that comply with network transmission protocols, supports uploading to the cloud, and forms the user's private cloud space. ; Another major feature is that it has improved the level of intelligence. IoT cameras can use artificial intelligence to perform deep learning, accurately identify and detect humanoid movement, crying, and other abnormal movements and noises, automatically track and shoot the movement trajectories of abnormal situations, and report to the user Push alarm information. The IoT camera can also use infrared night vision technology to automatically switch between day and night modes to achieve all-weather shooting; the IoT camera also adds a voice interaction function, which can realize two-way voice calls, and uses the built-in remote control module to realize other IoT intelligent Hardware controls.

　　In recent years, with the improvement of IoT and artificial intelligence technology, IoT cameras have been effectively integrated with other household products, forming many AIoT products based on visual technology, such as baby monitors, smart cat eyes, video door locks, and video pets Feeders, visual sweeping robots, etc., the application scenarios are constantly expanding.

　　In addition to home use, IoT cameras are gradually being used in small and microphysical stores, such as convenience stores, small supermarkets, and small restaurants. Use humanoid recognition, motion detection, and tracking technology to monitor the entry and exit of small and microphysical stores, and play an active role in preventing product loss, property theft, evidence tracing of accidents such as store disputes, and remote supervision and management of store staff.

　　With the characteristics of excellent performance and high overall cost-effectiveness, IoT cameras have shown explosive growth in the home market, and they can interact and integrate with other home AIoT products to meet the different convenience needs of consumers. In addition, IoT cameras have been used in small and microphysical stores. As their performance is further improved, they will gradually be extended to security fields such as communities, small factories, and smart retail, as well as broader application scenarios. The development prospects are broad.

　　IoT camera chip

　　The real scene captured by the camera lens is collected by the sensor as image data and transmitted to the IoT camera chip for processing:

　　(1) The image data is first processed by the ISP in the SoC chip, and image noise, brightness, chroma, and other aspects are optimized through a series of algorithms such as 3A algorithm, CFA interpolation, 2/3D denoising, and wide dynamic range so that the image can be Restoration and enhancement;

　　(2) Then the NPU in the SoC chip will perform intelligent analysis of human figures and faces on the restored and enhanced images;

　　(3) Finally, the audio and video encoder in the SoC chip compresses the processed audio and video data. The compressed audio and video code stream and intelligent analysis results are transmitted to the cloud platform through wireless communication, and interact with smart terminals such as mobile phones to achieve real-time Watch; or storage in the cloud, users can decode and watch at any time.

　　IoT cameras have experienced the development process of 1 million pixels (HD, 720P) and 2 million pixels (Full HD, 1080P). Currently, 3-4 million pixel IoT cameras have become mainstream products in the industry. With the popularization of 5G and Wi-Fi6 technologies, wireless communication transmission speeds will be greatly increased and data delays will be reduced, pushing video definition to ultra-high definition (4K, 8K) and further improving the clarity and user experience of home cameras. IoT cameras can also be integrated with more smart home products, creating more market demand. On the other hand, as IoT cameras develop towards ultra-high definition, the data transmission, storage and computing models of traditional security cameras will be changed. IoT camera chips can be embedded in various devices. The product styles of security cameras will be diversified, not only in the form of gun cameras and dome cameras, which will increase the penetration rate of IoT camera chips in security cameras. Therefore, IoT camera SoC chips with 4K and 8K ultra-high-definition resolution video encoding capabilities will develop rapidly.

　　IoT cameras have gradually developed from the initial image collection function to the ability to perform some basic recognition algorithm processing on the collected images. In recent years, as intelligent processing capabilities based on deep learning algorithms have begun to be integrated into IoT cameras, IoT cameras integrating artificial intelligence analysis capabilities will be an important development trend. Therefore, camera chips with intelligent image analysis algorithms and intelligent voice recognition are the future development direction. The intelligent development of IoT cameras includes two aspects: intelligent image processing and intelligent analysis: Intelligent image processing needs to enhance the intelligent processing capabilities of ISPs to ensure clear images in low illumination, wide dynamic range, and jittering environments.

　　Currently, mainstream IoT cameras on the market mainly record two-dimensional images of fixed scenes and receive sounds from fixed directions. Even if they add a rotation function or use a fisheye lens, they only increase the width of the viewing angle, and the recorded images are still two-dimensional. , the received sound direction is still fixed. AR technology and VR technology are referred to as XR technology. IoT cameras with XR technology can collect surrounding scenes through camera arrays or multiple cameras, and collect surrounding sounds through microphone arrays. Users can watch and listen from virtual angles and directions on ordinary monitors and speakers. content and interact in real-time. Therefore, in the future, IoT camera chips need to be able to support multiple camera interfaces and have technical capabilities such as multi-channel video processing, microphone array and far-field sound pickup, image splicing, distortion correction, and depth detection.

　　(2) Internet of Things application processor chip

　　IoT application processor products

　　①Building intercom

　　Building intercom systems refer to electronic systems used in residential and commercial buildings that have functions such as calling, intercom, and visibility, and can control unlocking. They can enable visitors, residents, and property management centers to communicate with each other in multi-story or high-rise buildings. , and control the entrance and exit channels of the community or unit. Typical application scenarios for building intercom systems include residential areas, villas, hospitals, banks, schools, and other areas. With the development of urbanization and smart cities, the transformation of new housing to high-end furnished houses and the renovation of old buildings will drive the development of the building intercom industry. The demand for the building intercom industry is sustainable and the market prospects are broad.

　　②Smart door lock Smart door lock is an intelligently improved product based on the traditional mechanical lock. It is more intelligent, safe, and convenient in terms of security, identification, and management. Smart door locks usually have functions such as fingerprint unlocking, password unlocking, Bluetooth unlocking, and the Internet of Things. They solve the core pain points of traditional mechanical locks such as people forgetting to bring keys, having trouble bringing keys, and losing keys. After several generations of product iterations, smart door locks Product categories continue to expand, prices become more diversified, and the product consumption environment gradually matures. It has been widely used in residential communities, villas, long-term rental apartments, and commercial stores, and is showing a rapid development trend.

　　IoT application chip

　　With the maturity of IoT, artificial intelligence, and big data technology, the functions of IoT intelligent hardware are gradually becoming more complex to meet people's increasingly diverse needs, which requires the development of IoT intelligent hardware master control SoC chips to a high degree of integration. Taking the smart door lock industry as an example, unlocking methods are gradually enriched. In addition to card swiping and fingerprint unlocking, Bluetooth, passwords, face recognition, and other methods have also been added. Therefore, chip manufacturers need to improve the integration level of chips to reduce the overall cost of downstream products while reducing the product development time of downstream customers.

　　Compared with traditional consumer electronics products, IoT industrial-grade chip products have a longer service life, a wider operating temperature range, and a more complex operating environment. They also need to have anti-static and anti-electromagnetic interference capabilities, which require IoT chips to be used in Reliability, and anti-interference capabilities are further improved.

　　Reducing chip power consumption has always been the development trend of IoT application processor chips. Using more advanced processes can reduce chip power consumption, but with the evolution of processes, leakage current problems have become increasingly prominent, requiring the adoption of low-power design technology from the chip design side. At the chip design level, multi-threshold design, multi-voltage design, dynamic frequency voltage scaling (DVFS), clock gating, power-aware memory, and power gating can be used to reduce chip power consumption. With the popularization and implementation of the "Dual Carbon Target" throughout society, low-power design will become an increasingly important development trend in the chip industry.

　　Three core indicators of chip design

　　The design of the chip mainly needs to consider three core indicators "PPA" to achieve the optimal design of the product. "PPA" refers to power consumption (PowerConsumption), performance (Performance), and area (Area, or die area, DieSize) respectively. Lower power consumption, stronger performance, and smaller die area are the goals pursued by chip research and development, but it is difficult to pursue the three indicators at the same time because the improvement of chip performance usually leads to an increase in area and power consumption. Therefore, the research and development process of each chip is actually to pursue the balance point of the above three core indicators.

　　The power consumption of the chip mainly includes two forms: dynamic power consumption and leakage power consumption. Dynamic power consumption is caused by transistor state switching; leakage power consumption is caused by quantum effects after the size of the transistor is reduced. In the context of "carbon peak" and "carbon neutrality", reducing the dynamic/leakage power consumption of chips has become a consensus in the chip industry. In addition to actively exploring new materials for chips, in the chip design stage, developing chips with more advanced processes, developing low-power chip design and verification processes, reasonable chip architecture, and self-developed power management IP can all help reduce chip power. Consumption.

　　Chip performance refers to the chip's ability to complete specific tasks. Different chips have different performance measurement indicators. For example, CPU is usually measured by MIPS, operating frequency, Cache capacity, number of instructions, threads, and other indicators; NPU is usually measured by OPS and hardware utilization. In the chip design stage, in addition to chip system architecture and computing power algorithm innovation, improving chip performance also requires the support of high-end EDA software and related equipment.

　　The area of a single wafer is fixed, and the current mainstream wafer areas are 8 inches and 12 inches. The smaller the area of a single die, the more chips can be produced from a single wafer. Therefore, while achieving the same functions and performance, the smaller the die area, the more obvious the cost advantage. Under the current manufacturing process conditions, the chip area can be reduced through chip architecture innovation, chip design optimization, and layout and wiring process processes.

　　Characteristics of SoC chip design

　　As a system-level chip, SoC chip has two significant characteristics: on the one hand, the transistor scale of SoC chip is huge, and the number of transistors on a chip ranges from millions to tens of billions; on the other hand, SoC can run and handle multiple tasks Complex systems, namely SoC chips, require collaborative design and development of software and hardware.

　　The huge hardware scale of SoC chips leads to their design usually using IP reuse. IP refers to the functional modules in SoC chips and has the characteristics of versatility, repeatability, and portability. During the SoC chip development process, R&D personnel can use IP to reduce duplication of work, shorten the R&D cycle, and reduce development risks. In addition, SoC chip design companies need to set up a software department to develop software systems supporting SoC chips.

　　SoC chip design is difficult and the system architecture is complex, involving the overall SoC chip architecture, various key tips such as central processing unit, audio and video codec, ISP, and wireless connection technology. SoC chip design companies have high requirements for the quality of R&D personnel. They need to have an R&D team with knowledge in signal processing, semiconductor physics, process design, circuit design, computer science, electronic information, and other professional fields. Comprehensive considerations need to be taken into consideration when designing. Multiple performance indicators, strong comprehensiveness, and difficult design.

　　The downstream application fields of SoC chips are vast, with many market segments and diversified characteristics. The update and iteration speed of downstream application products is relatively fast, so chip design companies need to continue to invest resources in deepening and optimizing chips, and constantly update and upgrade on the original basis. In addition, the maturity of AIoT technology has put forward higher requirements for intelligent computing power, power consumption, and integration of chips, which will further increase the complexity of SoC chip design.