(EVM6446)
I want to write an algorithm of video processing in DSP, but I don't know how to begin with: I have two questions:
1. Which system should I select: Linux or Windows? I think: CCS+Windows is better, is that right? Could you tell me if I use linux , which dev tools is fit for developing DSP side.
2.About Algorithm of video processing: for example: If I want to write an algorithm to detect the edge of an object, I think first I should fetch video data from a block of DDR2, because this block is just writen by VPFE, and I believe the data should be an array of one dimension. Second, I should ask DSP to deal with the array using the algorithm. Third, the processed array should be sent back to the block of DDR2, because at this time this block will be sent to VPBE.
To describe the content in details:
a). ARM side: VPFE writes video data in DDR2
b).DSP side: To fetch the data from DDR2
c) DSP side: Algorithm section( for example: edge detection etc.)
d) DSP side: Sent back processed video data to DDR2
e) ARM side: VPBE gets the data from DDR2 and shows them in LCD
I don't whether my thinking is right. Do you have any suggestions?
Thank you.
1) For developing DSP algorithm, CCS running under windows is your best choice. At the current time, CCS does not run under Linux; this is why we only include the tool chain (compiler, linker, DSP/BIOS...) as part of DVSDK installation rather than a full blown IDE such as CCS (which includes toolchain under Windows).
2) The scenario you described is exactly what the DVSDK software architecture does; therefore, the less of this software (hopefully just DSP algorithm) you have to write yourself, the better.
a) This is taken care of by the V4L2 (VPFE) Linux video driver running on ARM.
b) In this step, the DSP does not actually fetch the buffer, the ARM side application which opened the V4L2 driver in the step above, passes the DSP the buffer pointer via codec engine API (codec engine is framework developed by TI). On a related note, please note that buffers shared between ARM and DSP reside in CMEM (contiguous memory manager developed by TI), this is because unlike ARM, DSP does not have a virtual memory manager, hence assumes memory buffer is contiguous (easier on the DSP video algorithm developer); also note that only buffer pointers are passed back and forth between ARM and DSP since they both have access to CMEM space. CMEM does reside in DDR2 space.
c) DSP algorithm processes buffer. Please note that in order to fit into our codec engine framework, DSP algorithm must be XDM compliant. XDM is based on XDAIS which requires DSP algorithms ask for resources rather than take them directly; this enables DSP algorithms from different vendors to play nicely together. There is documentation included in DVSDK that details these requirements.
d) DSP algorithm returns processed buffer pointer to ARM side. At present, AR/Linux side makes blocking codec engine API call to DSP side which DSP eventually returns with processed buffer pointer; the actual buffer is in CMEM space which both ARM and DSP can both read and write to.
e) ARM side application can do anything it wants with the processed buffer (display it, store it in HDD, stream it to a network...). If you want to display, then Linux application would call on the Linux frame buffer driver (VPBE).
I hope this helps point you in the right direction; I would recommend you go over some of some of the codec engine documentation first (included in corresponding folder under DVSDK), and as you get closer to developing your algorithm maybe learn a bit more about XDAIS (documentation also included in DVSDK). Understanding these are key.
After reading Codec Engine documents, I have these questions:
1. Can I write an algorithm which is not about "compress or decompress" into Codec Engine? (I'm sure it is XDM-compliant). For example: an algorithm about "edge detection" or "Dilation and Erosion" processing.
2. An example: Codec Engine: CE_InstallationPath/packages/ti/sdo/ce/video/videnc.h XDAIS interface: XDAIS_path/packages/ti/xdais/dm/ividenc.h
Algorithm: CE_InstallationPath/examples/codecs/videnc_copy/videnc_copy.c
If I write an ARM application using VIDENC_process function which is in videnc.h, how does XDM know what I want to call is VIDENCCOPY_TI_process in videnc_copy.c ? Because I do not find any relation between videnc_copy.c and videnc.h except that both of them include the same interface ividenc.h.
3. I don't know whether it is neccessary to use thread (pthread.h) in ARM application code when the code need to use XDM algorithm. In "encodedecode" demo, I find that this code use thread to send data between "display thread" and "camera thread". In "video thread", the camera data is encoded by h264 algorithm.
1) Yes, you can create your own algorithm that does not involve "compression" or "decompression"; please see 'scale' example.
2) Each DSP algorithm is required to implement a minimum set of functions in common (see IALGFXNS definition in videnc_copy.c); codec engine expands on the set by requiring two additional functions (process and control). These functions are mapped (in a required order) to functions (which have more flexible naming requirements such as VIDENCCOPY_TI_process) within the DSP algorithm. Since each DSP algorithm needs to define these functions and register them with the framework, the one called by the framework depends on which algorithm is instantiated in the user application calling VIDENC_Process.
3) It is not necessary to create pthread to call codec engine; FYI, each Linux process (user program) inherently has at least one thread. We just chose to create multiple threads in our demos.
Hi Juan,
I feel that it is better to develop DSP algorithm in Linux than in Windows CCS. I try to revise some code in "videnc_copy" example in codecs, and then I compile it with XDC tools. After that, I apply this new algorithm in encodedecode demo. Everything sounds good.
Could you tell me:
1. Why did you say : it's better to develop DSP algorithm in windows CCS ? Because I feel linux is also support it very well.
2. I did not find any math lib to support my algorithm in DSP, for example: FFT or DFT etc. Could you tell me how to find this math lib, or , I have to code FFT(etc.) by myself. (in Linux system)
Could you give me a suggestion:
To be a Davinci engineer, what is most important? ( To try to be familiar with coding in Davinci OR To try to be familiar with circuit pad design, for example: PCB design )
Thanks.
Hi Lorry,
1.) The reason I suggested Windows CCS is because many customers prefer to use an IDE (e.g. CCS) during development/debugging of their DSP algorithm. That said, we do offer everything but the GUI IDE in Linux (DSP/BIOS, compiler, linker...), so if you are ok working via command line in Linux, than this is great. In addition, during integration phase, you will need to move over to Linux anyway, hence developing in Linux gives you a head start when you get to this step; you just give up a nice IDE (CCS) to debug your DSP algorithm.
2) I believe these functions are found under DSPLIB library: http://focus.ti.com/docs/toolsw/folders/print/sprc265.html
Additionally, you can see a list of all available DSP libraries at the following link; please note that DM644X is based on 64x+ DSP core; hence not all libraries listed apply: http://focus.ti.com/dsp/docs/dspfindtoolswresults.tsp?sectionId=3&tabId=1620&familyId=44&toolTypeId=44&go=Go
With regards to your question about what makes a good DaVinci engineer. I think both hardware and software design are important; however, I am a bit more biased toward the software side since most questions we get are software-based.
If I want to create a CE (lib in CCS) using CCS, do I need to write a cmd file? I think I do not need to do it, because CE is calling by an ARM application, so the embedded linux system has allocated memory address and it's length for application. a CE program does not need to consider it. Am I right?
Lorry,
I am not too familiar with DSP side of the equation, but I believe you will need a cmd file. Our codec libraries are created and tested on CCS all by themselves (no ARM code needed), thus this leads me to think that they need a cmd file. As a matter of fact, the ARM does not know anything about DSP memory space. The system designer role is the divide the memory into ARM, DSP, and CMEM(shared) memory space. CE Engine_open call simply programs hardware registers to take DSP out of reset and program a register to let hardware know where in memory to load DSP image (defined by system designer and specified in DSP server condifguration files). I think if we start looking thru some of the codec engine codecs, servers, and apps files, you will see files resembling linker cmd files (still needed).
Anyway, I know this is not a clearly detailed answer, but hopefully I have given you enough info to get a little further.
A Codec Engine server (.x64p) is actually a full blown executable and not just a library, as Juan is saying Linux is not allocating space for the DSP application to run, that is done by the user in advance, the Link application running in Linux sitting under the Codec Engine framework APIs will load a DSP executable to the appropriate area of DDR and set the DSP off running it. When you build a Codec Engine server you do need a linker command file because you are making an executable, in a typical build environment you use DSP/BIOS to handle the generation of the linker command file for you, so it exists but you do not write it yourself. You can adjust how the memory map is to be setup in your BIOS configuration file (.tcf) to determine how the linker command file is generated, since you are doing this in CCS you can even open up the tcf file in your project in a GUI form to modify these sorts of things.
What you are proposing would be nice, but unfortunately the Codec Engine ARM side framework is not intelligent enough to dynamically relocate and execute a DSP library, since it would have to go through part of the build process with the relocation (which has to be outside of Linux memory anyway as the DSP has no MMU) this would probably slow things down too much to be worth the integration ease it would add.
If I just want to build a lib (a64P) using CCS, do I need a cmd file in CCS?
I think another mothod to build an x64p file is to build a lib (a64P) in CCS, and then copy it to Linux system and rename it (*.lib ---> *.a64P). Finally, I use xdc tools to build a x64P file in Linux system. I think this way to build an x64P file is easier than building x64P directly in CCS.
This is not my area of expertise, but I do not believe you need to define a cmd file if you are building a codec library; as a matter of fact, I imagine it would be against XDAIS rules to do so as a system integrator should be able to take your codec library and build a DSP server to run in a DSP memory space of his choosing (the integrator defines the memory map). However, I imagine at the library level you still need to define the appropriate memory segment names (not sure is standardized memory segments to be used are pre-defined as part of XDAIS) so that they match those of the codec engine framework. Anyway, I know I am not being as specific as you would probrably like, but as I mentioned, this is not my area of expertise. I just wanted to give you my opinion in hopes that you can make further progress; if you continue to have trouble, please drop us another note and I will see if I can get the right person next week to chime in.
Thanks Juan, I'll try what I've written in my post, if I have any questions, I'll post them here. :)
1.
I've tried to compile an achive file in CCS (for example: videnc_copy.lib) in windows, and then I rename it (videnc_copy.a64P) in lib folder in Linux. And then I compile an codec server and copy it to "/home/usraccount/workdir/filesys/opt/dvevm". I want to know whether my codec can work well, but I failed.Sequence:a) I compile videnc_copy.lib in CCS, and I'm sure that all the interfaces have been realized.
b) I copy videnc_copy.lib to /home/usracc/dvevm_x_xx/codec_engine_x_xx/examples/codecs/videnc_copy/lib , and I rename this lib file to videnc_copy.a64P.
c) In .../examples/servers/video_copy/ folder, I recompile codec server file and copy it to .../workdir/filesys/opt/dvevm.
d) I want to test whether videnc_copy can work well.
After checking, I find that the videnc works, but it is not the code that I compile in CCS, it works as the original videnc_copy.a64P in videnc_copy example in Linux. It sounds that the videnc_copy.lib does not affect codec server (video_copy.x64P).
So I guess, the new videnc_copy.a64P( renamed from videnc_copy.lib) is not compiled when generating video_copy.x64P.
2.I find a tool named RTSC_Codec_And_Server_Package_Wizards , It can help us to compile codec server in windows without using CCS. Could the codec generated by RTSC debug in CCS?
3.If I run a program in target, an error shows " EVM # /dev/dsplink: No such file or directory" debugging at "Engine_open", finally I find that I forget to run "./loadmodules.sh" at the beginning, I find the answer here titled "problem:Getting error when running sample applications on Linux". I think there should be another "possible cause", although it is not related with my error.
When compiling an codec server, I know it need lots of achieve files ( osal_dsplink_bios.a64P, bioslog.a64P,....etc.), but if "dsplink.lib" or "osal_dsplink_bios.a64P" is not compiled by XDC when generating codec server, I wonder this error can also occur when a program runs on target board.
So I think my description can be "possible cause 3". right?
Thank you for the detailed explanation. It is my understanding that the codec library is pre-compiled ahead of time to produce .a64P library file; this library file is linked into the DSP Server when you build DSP Server. That said,
1) When you build your codec server thru RTSC wizard tool (or possibly other ways), you should be able to debug it using CCS provided it is a DSP server (most common); FYI, codec engine architecture allows us to build codec servers that run on ARM/Linux as well, but CCS would not be much help there.
2) DSPLINK resides partially on ARM and partially on DSP. Therefore, normally DSP portion of DSPLINK is linked into final DSP codec server executable (no need to have .lib or .a64P libraries in target) and ARM portion of DSPLINK is loaded via loadmodules.sh.
Let me know if this helps
Thank Juan,
For 1)
Actually, I think I don't grasp the "spirit" on how to write and debug codec server in CCS yet, even if I've known the structure of XDM interface and other rules of codec. So I chose RTSC to help me to compile. I really want to read a document to instruct me how to debug codec in CCS, for example: one of your documents on how to debug an ARM program using DDD. This document helps me a lot, because it describes all the sequence step by step, but I still do not find any documents about makeing or debugging codec in CCS.
If this type of document is not existed, I think I should read "makefile" in codec engine or codec server carefully to learn the entire process on how to compile an codec engine and server. And then I think that will help me to build codec in CCS in Windows.
For 2)
That means," EVM # /dev/dsplink: No such file or directory" tells us that the ARM program needs dsplink driver or can not find dsplink driver, is that right? But this error message occurs when I debug on "Engine_open" line. Is the error not related with Engine server on DSP?
ALL CONTENT AND MATERIALS ON THIS SITE ARE PROVIDED "AS IS". TI AND ITS RESPECTIVE SUPPLIERS MAKE NO REPRESENTATIONS ABOUT THE SUITABILITY OF THESE MATERIALS FOR ANY PURPOSE AND DISCLAIM ALL WARRANTIES AND CONDITIONS WITH REGARD TO THESE MATERIALS, INCLUDING BUT NOT LIMITED TO, ALL IMPLIED WARRANTIES AND CONDITIONS OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT OF ANY THIRD PARTY INTELLECTUAL PROPERTY RIGHT. NO LICENSE, EITHER EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, IS GRANTED BY TI. USE OF THE INFORMATION ON THIS SITE MAY REQUIRE A LICENSE FROM A THIRD PARTY, OR A LICENSE FROM TI.
Content on this site may contain or be subject to specific guidelines or limitations on use. All postings and use of the content on this site are subject to the Terms of Use of the site; third parties using this content agree to abide by any limitations or guidelines and to comply with the Terms of Use of this site. TI and its suppliers reserve the right to make corrections, deletions, modifications, enhancements, improvements and other changes to the content and materials, its products, programs and services at any time or to move or discontinue any content, products, programs, or services without notice.